Skip to content


Where Would You Like to See the Next Xen Project User Summit Held?

In 2013, we held the first major Xen event aimed specifically at users: the Xen Project User Summit. In 2014, we want to do it again — but where and when?

The Xen Project wants to hold its second Xen Project User Summit.  We’d like to hold it somewhere which is accessible by a large percentage of our user community.  And we’d like to schedule it at a time which makes sense, possibly in coordination with some existing conference.

We need your help to pick the time and place.  Give us your preferences in a very quick 2 minute survey found here:

https://www.surveymonkey.com/s/YJQCHJ6

It’s very quick and easy to do.  And you may just find that the next User Summit is too convenient for you to pass up.

Posted in Announcements, Community, Events, Xen Summit.

Tagged with , .


What is the ARINC653 Scheduler?

The Xen ARINC 653 scheduler is a real time scheduler that has been in Xen since 4.1.0.  It is a cyclic executive scheduler with a specific usage in mind, so unless one has aviation experience they are unlikely to have ever encountered it.

The scheduler was created and is currently maintained by DornerWorks.

Background

The primary goal of the ARINC 653 specification [1] is the isolation or partitioning of domains.  The specification goes out of its way to prevent one domain from adversely affecting any other domain, and this goal extends to any contended resource, including but not limited to I/O bandwidth, CPU caching, branch prediction buffers, and CPU execution time.

This isolation is important in aviation because it allows applications at different levels of certification (e.g. Autopilot – Level A Criticality, In-Flight Entertainment – Level E Criticality, etc…) to be run in different partitions (domains) on the same platform.  Historically to maintain this isolation each application had its own separate computer and operating system, in what was called a federated system.  Integrated Modular Avionics (IMA) systems were created to allow multiple applications to run on the same hardware.  In turn, the ARINC653 specification was created to standardize an Operating System for these platforms.  While it is called an operating system and could be implemented as such, it can also be implemented as a hypervisor running multiple virtual machines as partitions.  Since the transition from federated to IMA systems in avionics closely mirrors the transition to virtualized servers in the IT sector, the latter implementation seems more natural.

Beyond aviation, an ARINC 653 scheduler can be used where temporal isolation of domains is a top priority, or in security environments with indistinguishability requirements, since a malicious domain should be unable to extract information through a timing side-channel.  In other applications, the use of an ARINC 653 scheduler would not be recommended due to the reduced performance.

Scheduling Algorithm

The ARINC 653 scheduler in Xen provides the groundwork for the temporal isolation of domains from each other. The domain scheduling algorithm itself is fairly simple:  a fixed predetermined list of domains is repeatedly scheduled with a fixed periodicity resulting in a complete and, most importantly, predictable schedule.  The overall period of the scheduler is know as a major frame, while the individual domain execution windows in the schedule are know as minor frames.

Major_Minor_Frame

As an example, suppose we have 3 domains all with periods of 5, 6, 10 ms and worst case running times respectively of 1 ms, 2 ms, and 3 ms.  The major frame is set to the least common multiple of these periods (30 ms) and minor frames are selected so that the period, runtime, and deadline constraints are met.  One resulting schedule is shown below, though there are other possibilities.

ExampleSchedule

The ARINC 653 scheduler is only concerned with the scheduling of domains. The scheduling of real-time processes within a domain is performed by that domain’s process scheduler.  In a compliant ARINC 653 system, these processes are scheduled using a fixed priority scheduling algorithm, but if ARINC 653 compliance is not a concern any other process scheduling method may be used.

Using the Scheduler

Directions for using the scheduler can be found on the Xen wiki at ARINC653 Scheduler. When using the scheduler, the most obvious effect will be that the cpu usage and execution windows for each domain will be fixed regardless of whether the domain is performing any work.

Currently multicore operation of the scheduler is not supported.  Extending the scheduling algorithm to multiple cores is trivial, but the isolation of domains in a multicore system requires a number of mitigation techniques not required in single-core systems.[2]

References

[1] ARINC Specification 653P1-3, “Avionics Application Software Standard Interface Part 1 – Required Services” November 15, 2010

[2] EASA.2011/6 MULCORS – Use of Multicore Processors in airborne systems

Posted in Xen Development.

Tagged with , .


Announcing the 1.0 release of Mirage OS

We’re very pleased to announce the release of Mirage OS 1.0. This is the first major release of Mirage OS and represents several years of development, testing and community building. You can get started by following the install instructions and creating your own webserver to host a static website! Also check out the release notes and download page.

What is Mirage OS and why is it important?

Most applications that run in the cloud are not optimized to do so. They inherently carry assumptions about the underlying operating system with them, including vulnerabilities and bloat.

Compartmentalization of large servers into smaller ‘virtual machines’ has enabled many new businesses to get started and achieve scale. This has been great for new services but many of those virtual machines are single-purpose and yet they contain largely complete operating systems which typically run single applications like web-servers, load balancers, databases, mail servers and similar services. This means a large part of the footprint is unused and unnecessary, which is both costly due to resource usage (RAM, disk space etc) and a security risk due to the increased complexity of the system and the larger attack surface.

Cloud OS Diagram

On the left, you see a typical application stack run in the cloud today. Cloud Operating systems such as MirageOS remove the Operating System and replace it with a Language Runtime that is designed to cooperate with the Hypervisor.

Mirage OS is a Cloud Operating System which represents an approach where only the necessary components of the operating system are included and compiled along with the application into a ‘unikernel’. This results in highly efficient and extremely lean ‘appliances’, with the same or better functionality but a much smaller footprint and attack surface. These appliances can be deployed directly to the cloud and embedded devices, with the benefits of reduced costs and increased security and scalability.

Some example use cases for Mirage OS include: (1) A lean webserver, for example the openmirage.org, website is about 1MB including all content, boots in about 1 second and is hosted on Amazon EC2. (2) Middle-box applications such as small OpenFlow switches for tenants in a cloud-provider. (3) Easy reuse of the same code and toolchain that create cloud appliances to target the space and memory constrained ARM devices.

How does Mirage OS work?

Mirage OS works by treating the Xen hypervisor as a stable hardware platform and using libraries to provide the services and protocols we expect from a typical operating system, e.g. a networking stack. Application code is developed in a high-level functional programming language OCaml on a desktop OS such as Linux or Mac OSX, and compiled into a fully-standalone, specialized unikernel. These unikernels run directly on Xen hypervisor APIs. Since Xen powers most public clouds such as Amazon EC2, Rackspace Cloud, and many others, Mirage OS lets your servers run more cheaply, securely and faster on those services.

Mirage OS is implemented in the OCaml language, with 50+ libraries which map directly to operating system constructs when being compiled for production deployment. The goal is to make it as easy as possible to create Mirage OS appliances and ensure that all the things found in a typical operating system stack are still available to the developer. Mirage OS includes clean-slate functional implementations of protocols ranging from TCP/IP, DNS, SSH, OpenFlow (switch/controller), HTTP, XMPP and Xen Project inter-VM transports. Since everything is written in a single high-level language, it is easier to work with those libraries directly. This approach guarantees the best possible performance of Mirage OS on the Xen Hypervisor without needing to support the thousands of device drivers found in a traditional OS.

Bind 9 vs. Mirage OS throughput comparison

Performance comparison of Bind 9 vs. a DNS server written in Mirage OS.

An example of a Mirage OS appliance is a DNS server and below is a comparison with one of the most widely deployed DNS servers on the internet, BIND 9. As you can see, the Mirage OS appliance outperforms BIND 9 but in addition, the Mirage OS VM is less than 200kB in size compared to over 450MB for the BIND VM. Moreover, the traditional VM contains 4-5 times more lines of code than the Mirage implementation, and lines of code are often considered correlated with attack surface. More detail about this comparison and others can be found in the associated ASPLOS paper.

For the DNS appliance above, the application code was written using OCaml and compiled with the relevant Mirage OS libraries. To take full advantage of Mirage OS it is necessary to design and construct applications using OCaml, which provides a number of additional benefits such as type-safety. For those new to OCaml, there are some excellent resources to get started with the language, including a new book from O’Reilly and a range of tutorials on the revamped OCaml website.

We look forward to the exciting wave of innovation that Mirage OS will unleash including more resilient and lean software as well as increased developer productivity.

Posted in Announcements.

Tagged with .


Xen on ARM and the Device Tree vs. ACPI debate

ACPI vs. Device Tree on ARM

Some of you may have seen the recent discussions on the linux-arm-kernel mailing list (and others) about the use of ACPI vs DT on the ARM platform. As always LWN have a pretty good summary (currently subscribers only, becomes freely available on 5 December) of the situation with ACPI on ARM.

Device Tree (or DT) and Advanced Configuration & Power Interface (or ACPI) are both standards which are used for describing a hardware platform e.g. to an operating system kernel. At their core both technologies provide a tree like data structure containing a hierarchy of devices and specifying what type they are and a set of “bindings” for that device. A binding is essentially a schema for specifying I/O regions, interrupt mappings, GPIOs and clocks etc.

For the last few years Linux on ARM has been moving away from hardcoded “board files” (a bunch of C code for each platform) towards using Device Tree instead. In the ARM space ACPI is the new kid on the block and has many unknowns. Given this the approach to ACPI which appears to have been reached by the Linux kernel maintainers, which is essentially to wait and see how the market pans out, seems sensible.

On the Xen side we started the port to ARM around the time that Linux’s transition from board files to Device Tree was starting and made the decision early on to go directly to device tree (ACPI wasn’t even on the table at this point, at least not publicly). Xen DT to discover all of the hardware on the system, both that which it intends to use itself and that which it intends to pass to domain 0. As well as consuming DT itself Xen also creates a filleted version of the host DT which it passes to the domain 0 kernel. DT is simple and yet powerful enough to allow us to do this relatively easily.

DT is also used by some of the BSD variants in their ARM ports as well.

My Position as Xen on ARM Maintainer

The platform configuration mechanism supported by Xen on ARM today is Device Tree. Device Tree is a good fit for our requirements and we will continue to support it as our primary hardware description mechanism.

Given that a number of operating system vendors and hardware vendors care about ACPI on ARM and are pushing hard for it, especially in the ARM server space, it is possible, perhaps even likely, that we will eventually find ourselves needing support ACPI as well. On systems which support both ACPI and DT we will continue to prefer Device Tree. Once ARM hardware platforms that only support ACPI are available, we will obviously need to support ACPI.

The Xen Project works closely with the Linux kernel and other open source upstreams as well as organisations such as Linaro. Before Xen on ARM can support ACPI I would like see it gaining some actual traction on ARM. In particular I would like to see it get to the point where it has been accepted by the Linux kernel maintainers. It is clearly not wise for Xen to be pioneering the use of ACPI before to it becoming clear whether or not it is going to gain any traction in the wider ecosystem.

So if you are an ARM silicon or platform vendor and you care about virtualization and Xen in particular, I encourage you to provide a complete device tree for your platform.

Note that this only applies to Xen on ARM. I cannot speak for Xen on x86 but I think it is pretty clear that it will continue to support ACPI so long as it remains the dominant hardware description on that platform.

It should also be noted that ACPI on ARM is primarily a server space thing at this stage. Of course Xen and Linux are not just about servers: both communities have sizable communities of embedded vendors (on the Xen side we had several interesting presentations at the recent Xen Developer Summit on embedded uses of Xen on ARM). Essentially no one is suggesting that the embedded use cases should move from DT to ACPI and so, irrespective of what happens with ACPI, DT has a strong future on ARM.

ACPI and Type I Hypervisors

Our experience on x86 has shown that the ACPI model is not a good fit for Type I hypervisors such as Xen, and the same is true on ARM. ACPI essentially enforces a model where the hypervisor, the kernel, the OSPM (the ACPI term for the bit of an OS which speaks ACPI) and the device drivers all must reside in the same privileged entity. In other words it effectively mandates a single monolithic entity which controls everything about the system. This obviously precludes such things as dividing hardware into that which is owned and controlled by the hypervisor and that which is owned and controlled by a virtual machine such as dom0. This impedance mismatch is probably not insurmountable but experience with ACPI on x86 Xen suggests that the resulting architecture is not going to be very agreeable.

UEFI

Due to their history on x86 ACPI and UEFI are often lumped together as a single thing when in reality they are mostly independent. There is no reason why UEFI cannot also be used with Device Tree. We would expect Xen to support UEFI sooner rather than later.

Posted in Thought Leadership, Uncategorized, Xen Hypervisor.

Tagged with , , , , .


RT-Xen: Real-Time Virtualization in Xen

RT-XenThe researchers at Washington University in St. Louis and University of Pennsylvania are pleased to announce, here on this blog, the release of a new and greatly improved version of the RT-Xen project. Recent years have seen increasing demand for supporting real-time systems in virtualized environments (for example, the Xen-ARM projects and several other real-time enhancements to Xen), as virtualization enables greater flexibility and reduces cost, weight and energy by breaking the correspondence between logical systems and physical systems. As an example of this, check out the video below from the 2013 Xen Project Developer Summit

The video describes how Xen could be used in an in-vehicle infotainement system.

In order to combine real-time and virtualization, a formally defined real-time scheduler at the hypervisor level is needed to provide timing guarantees to the guest virtual machines. RT-Xen bridges the gap between real-time scheduling theory and the virtualization technology by providing a suite of multi-core real-time schedulers to deliver real-time performance to domains running on the Xen hypervisor.

Background: Scheduling in Xen

In Xen, each domain’s core is abstracted as a Virtual CPU (VCPU), and the hypervisor scheduler is responsible for scheduling VCPUs. For example, the default credit scheduler would assign a weight per domain, which decides the proportional share of CPU cycles that a domain would get. The credit scheduler works great for general purpose computing, but is not suitable for real-time applications due to the following reasons:

  1. There is no reservation with credit scheduler. For example, when two VCPUs runs on a 2 GHz physical core, each would get 1 GHz. However, if another VCPU also boots on the same PCPU, the resource share shrinks to 0.66 GHz. The system manager have to carefully configure the number of VMs/VCPUs to ensure that each domain get an appropriate amount of CPU resource;
  2. There is little timing predictability or real-time performance provided to the VM. If a VM is running some real-time workload (video decoding, voice processing, and feedback control loops) which are periodically triggered and have a timing requirement — for example, the VM must be scheduled every 10 ms to process the data — there is no way the VM can express this information to the underlying VMM scheduler. The existing SEDF scheduler can help with this, but it has poor support for multi-core.

RT-Xen: Combining real-time and virtualization

RT-Xen aims to solve this problem by providing a suite of real-time schedulers. The users can specify (budget, period, CPU mask) for each VCPU individually. The budget represents the maximum CPU resource a VCPU will get during a period; the period represents the timing quantum of the CPU resources provided to the VCPU; the CPU mask defines a subset of physical cores a VCPU is allowed to run. For each VCPU, the budget is reset at each starting point of the period (all in milliseconds), consumed when the VCPU is executing, and deferred when the VCPU has budget but no work to do.

Within each scheduler, the users can switch between different priority schemes: earliest deadline first (EDF), where VCPU with earlier deadline has higher priority; or rate monotonic (RM), where VCPU with shorter period has higher priority. As a results, not only the VCPU gets a resource reservation (budget/period), but also an explicit timing information for the CPU resources (period). The real-time schedulers in RT-Xen delivers the desired real-time performance to the VMs based on the resource reservations.

To be more specific, the two multi-core schedulers in RT-Xen are:

  • RT-globalwhich uses a global run queue to hold all VCPUs (in runnable state). It is CPU mask aware, and provides better resource utilization, as VCPU can migrate freely between physical cores (within CPU mask)
  • RT-partition: which uses a run queue per physical CPU. In this way, each physical CPU only looks at its own run queue to make scheduling decisions, which incurs less overhead and potentially better cache performance. However, load-balancing between physical cores is not provided in the current release.

Source Code and References

The developers of RT-Xen are looking closely at how to integrate both schedulers into the Xen mainstream. In the meantime, please check out publications at [Tech Report’13] [RTAS’12] [EMSOFT’11] (the latter two focus on single-core case) and source code.

Posted in Partner Announcements, Xen Development, Xen Hypervisor.

Tagged with , , , , , .


Xen Project Well Represented at SUSECon and openSUSE Summit

What do a chameleon, a panda, and a mouse have in common?  More than you might imagine, unless you were present at SUSECon and the openSUSE Summit at the Walt Disney Coronado Springs resort last week in Florida.  During the week, it was clear that the SUSE chameleon and Xen panda could happily coexist in Mickey Mouse’s home turf.

Geeko, the openSUSE Chameleon

I had the opportunity to attend and speak at both conferences. The week started with 3 and a half days of SUSECon, an event dedicated to SUSE’s commercial Linux products, and finished with 2 days of openSUSE Summit, which celebrates the Open Source distribution. Key people from both worlds filled the halls, and the schedule boasted an excellent assortment of talks.

At least a half dozen different sessions included some Xen-related content at SUSECon, not to mention additional sessions at openSUSE Summit.  Both the Open Source openSUSE distribution and the commercial SUSE Linux Enterprise Server (SLES) boast support for multiple virtualization engines (Xen, KVM, Linux containers, etc.).  While this might not seem too significant, it is a refreshing departure from many companies in the industry which insist on hawking their preferred virtualization technology. On platforms where hypervisors have equal footing, Xen has the opportunity to shine — and usually does.

Continued…

Posted in Community, Events.

Tagged with , , .


Xen Project Developer Summit Slides and Videos are Live

Most of the Xen Project Developer Summit Videos and Presentations are now live on Xenproject.org. I wanted to thank all our speakers and attendees for making this year’s Xen project Developer Summit a success and am looking forward to next year’s event.

A few Highlights


Will Auld from Intel explains how any unmodified OS could be used as HVM Dom0 in Xen


Luwei Cheng talks about his research on TCP performance in VMs

Lovene Bhatia from Samsung shows two Android VMs running on a Nexus 10

Note that we will be running a series of in-depth articles on some of the topics that have been discussed at the Xen Project Developer and User Summits over the coming months. Some presentations, videos and Bof notes are still missing. Please check the Dev Summit page occasionally for updates.

Posted in Events.

Tagged with , .


Xen Project will co-host FOSDEM’14 Virtualisation and IaaS DevRoom

fosdem14

We are pleased to announce that the Xen Project will co-host a two-day Virtualisation and IaaS DevRoom at FOSDEM’14, alongside Redhat and the Openstack Foundation.

Call for Papers: Closes December 1st

The Call for Papers for the DevRoom will be open until December 1st. The scope for this devroom is open source, openly-developed projects in the areas of virtualisation and IaaS type clouds, ranging from low level to data center, up to cloud management platforms and cloud resource orchestration.

Sessions proposals should always target a developer audience and proposals for collaborative sessions that would be appealing to developers from multiple projects will be preferred. FOSDEM is about collaboration and unique in that it brings developers from many different projects and technologies together.

We (the DevRoom organizers) are particularly interested in the following themes:

  • Innvovation in low level virtualisation
  • New features in classic and container-based virtualisation technologies
  • New use cases for virtualisation, such as virtualisation in mobile, automotive and embedded in general
  • Other resource virtualisation technologies: networking, storage, … and their integration into other projects
  • Deep technical dives into specific IaaS or virtualisation management project’s features
  • Relationship between IaaS projects and specific dependencies (not just virtualisation)
  • Integration and development leveraging solutions from multiple projects

More information about the CfP and how to submit a proposal here.

FOSDEM

The Xen Project has had a strong presence at FOSDEM in the last two years and FOSDEM is always a lot of fun and very productive. It is entirely volunteer organized and free to attend, which makes the event different from many other open source events. Why not check out our event report from last year and submit a talk proposal for our DevRoom.

Posted in Events.


My first Xen Developer Summit: expectations vs reality

Last week I was in Edinburgh to attend my first Xen Project Developer Summit. It turned out quite different from my expectations, so I thought I’d share a bit about my experience.

When I first saw the call for participation I wasn’t sure that I had anything sufficiently relevant to contribute that would be of interest to the community. But I decided there wouldn’t be any harm in submitting a proposal for a talk, and it came as a bit of a surprise when I learned that it had been accepted.

My day job is to work on the performance and scalability of XenServer, Citrix’s enterprise-grade virtualisation platform built on Xen. The XenServer engineering team had recently made great strides in improving VM density scalability so it was natural for my talk to be on this topic. The reason for my uncertainty over the relevance of this talk was that it was all about stuff built around the hypervisor and not so much about the hypervisor itself. Evidently I was wrong to assume that this wouldn’t be of interest.

So who’s in the Xen community?

Once my talk had been accepted, I looked forward to attending and seeing what the Xen community looks like. My expectation was that it would be a meeting of the regulars on the xen-devel mailing list verbalising the same kind of discussions as are typically held on there. This wouldn’t have been a bad thing, but I’d have been an outsider on the sidelines, being unable to give or take a lot from this.

But I was wrong. The attendees constituted a surprisingly broad group of people. There were some from academia, others from industry. There were some doing research and others commercialising it. There were some who were hacking on the Xen hypervisor itself and others involved with construction of products built on top of it.

Luwei Cheng talks about his research on TCP performance in VMs

Luwei Cheng talks about his research on TCP performance in VMs

What about the talks?

Compared to similar-sized conferences I’ve attended in other fields of technology in the past, the quality of talks delivered at the Xen Developer Summit was generally very high; I was impressed. And there was an excellent range of presentations, covering a good variety of topics. I was surprised to find that there were many that were relevant to my own work. I was especially interested by talks on:

But it wasn’t all about performance! For those with interest in other topics, there was plenty of content concerning Xen-on-ARM, Android support, GPU integration, security, microkernel VMs, and a lot more.

So I very much enjoyed my first Xen Developer Summit. There was a real buzz around the Xen community. I learned a lot and made some great contacts, and hope that I’ll be able to return to future Summits.

Posted in Events.

Tagged with , .


Announcing the Release of Xen Project 4.3.1

We are pleased to announce the release of Xen 4.3.1.  The is the latest point release in the Xen 4.3 series of releases.

Downloads:

This is available immediately from its git repository:

xenbits.xenproject.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.3 (tag RELEASE-4.3.1).

It is also available for download from the XenProject.org website:

xenproject.org/downloads/xen-archives/supported-xen-43-series/xen-431.html

Fixes:

This fixes the following critical vulnerabilities:

  • CVE-2013-1922 / XSA-48 qemu-nbd format-guessing due to missing format specification
  • CVE-2013-2007 / XSA-51 qemu guest agent (qga) insecure file permissions
  • CVE-2013-1442 / XSA-62 Information leak on AVX and/or LWP capable CPUs
  • CVE-2013-4355 / XSA-63 Information leaks through I/O instruction emulation
  • CVE-2013-4356 / XSA-64 Memory accessible by 64-bit PV guests under live migration
  • CVE-2013-4361 / XSA-66 Information leak through fbld instruction emulation
  • CVE-2013-4368 / XSA-67 Information leak through outs instruction emulation
  • CVE-2013-4369 / XSA-68 possible null dereference when parsing vif ratelimiting info
  • CVE-2013-4370 / XSA-69 misplaced free in ocaml xc_vcpu_getaffinity stub
  • CVE-2013-4371 / XSA-70 use-after-free in libxl_list_cpupool under memory pressure
  • CVE-2013-4375 / XSA-71 qemu disk backend (qdisk) resource leak
  • CVE-2013-4416 / XSA-72 ocaml xenstored mishandles oversized message replies

We recommend all users of the 4.3 stable series to update to this latest point release.

Among the bug fixes and improvements (around 80 since Xen 4.3.0):

  • Adjustments to XSAVE management
  • Bug fixes to nested virtualization
  • Bug fixes for other low level system state handling
  • Bug fixes to the libxl tool stack

Posted in Announcements, Xen Hypervisor.

Tagged with , .