Virtualization Blog

Discussions and observations on virtualization.

XenServer Status – November 2013

The progress towards fulfilling the goal of making XenServer a proper open source project continues, but this month much of the work isn’t visible yet.  The big process improvements will hopefully be unveiled in late December or early January when we get our long needed wiki and defect trackers online.  The logical question of course is why it’s taking so long to get them out there.  After all we obviously do have the content, so why not just make it all public and be done?  Unfortunately, there is no magic wand to remove customer sensitive information, ensure that designs linked to closed source development on other Citrix products, or information provided to Citrix by partners under NDA isn’t accidentally made public.  Its painstaking work and we want to get it right.

In terms of partner announcements, we’ve been focusing on the NVIDIA vGPU work, as well as security efforts.

-          “Kaspersky trusted status” awarded to XenServer Windows Tools:

-          SAP 3D Enterprise on XenDesktop on XenServer powered by NVIDIA GRID:

-          The XenServer HCL has been expanded to include new servers from HP, Hitachi, Supermicro, Huawei, Lenovo and Fujitsu, storage devices from QNAP, Nexsan and Hitachi Data Systems, storage adapters from IBM and QLogic plus two CNAs from Emulex.

When I posted the project status last month, we had some significant gains, and this month is no different.  Compared to October:

-          Unique visitors were up 30% to 34,000

- page views were up 21% to over 110,000

-          Downloads of the XenServer installer were up by 7,000

-          We had over 110 commits to the XenServer repositories.

What’s most interesting about these stats isn’t the growth, which I do love, but that we’re getting to a point where the activity level is starting to feel right for a project of our maturity.  Don’t get me wrong, I still am looking for lots more growth, but I’m also looking for sustained activity.  That’s why I’m looking more at how XenServer interacts with its community, and what can be done to improve the relationship.  In my Open@Citrix blog, I asked the question “What kind of community do you want?”.  In my mind, everyone has a voice; it’s just up to you to engage with us.  I’d like to hear what you want from us, and that’s both the good and the bad.  If you have a community you’d like us to be involved with, I’d also like to hear about that too. 

Here is how I define the XenServer community:

The XenServer community is an independent group working to common purpose with a goal of leveraging each other to maximize the success of the community.  Members are proud to be associated with the community.


We all have a role to play in the future success of XenServer, and while I have the twitter handle of @XenServerArmy, I view my role as supporting you.  If there is something which is preventing you from adopting XenServer, or being as successful with XenServer as you intended, I want to know.  I want to remove as many barriers to adopting XenServer as I can, and I am your voice within the XenServer team at Citrix.  Please be vocal in your support, and vocal with what you need.

Recent Comments
srinivas j
XenServer status? Please post or update the status of XenServer roadmap
Sunday, 05 January 2014 03:31
srinivas j
currently hold XenServer licenses for 10+ hosts and are eagerly waiting for any updates to XenServer roadmap..
Sunday, 05 January 2014 03:32
Continue reading
12543 Hits

I am going to write some Chinese language post



I will teach how to use XenServer.

Recent Comments
Tim Mackey
Thank you Martin. I agree the market is far from universal and welcome your contributions. It also reminds me that we have a Sim... Read More
Friday, 22 November 2013 22:13
Jie Zhang
It's great for Chinese XenServer user!
Thursday, 28 November 2013 14:33
Continue reading
24863 Hits

XenServer at the Xen Project Developer Summit

Last month Edinburgh was host to the latest Xen Project Developer Summit.  For those who attended, and for those who couldn't make it this time, the session recordings and presentations have just been posted.  The event saw content ranging from Xen on ARM and Xen in automotive infotainment, through to deeper discussions on fault tolerant system design and video display handling from a significant number of non-Citrix presenters.  Of course there were several XenServer related sessions, and I’m including the abstracts and links for them below.

“Unlimited” Event Channels

David Vrabel, Citrix

Event Channels are Xen's mechanism for paravirtualized interrupts. These were limited to only 4096 which then limits the number of guests that a host may support to around 300 to 500. This presentation will give a brief introduction to event channels, a detailed look at the new, innovative FIFO-based event channel ABI that increases the limit to over 100,000 as well as having several other useful features (e.g., multiple priorities). Some of the key performance measurements of the new ABI will be shown.

View presentation          View session recording

Test-as-a-Service and XenRT

Alex Brett, Citrix

In this presentation, Alex Brett will show how Citrix has constructed a Test-as-a-Service environment which is used by the wider XenServer engineering team, highlighting the benefits the approach provides, together with an introduction to the (recently open sourced) XenRT automation framework which powers it, and discuss how this could be applied within the Xen Project community.

View presentation          View session recording

Increasing XenServer’s VM Density

Jonathan Davies, Citrix

As the number of CPU cores in server-class hardware increases, the demand on a virtualisation platform increases for greater VM density. Most commercial virtualisation platforms now support several hundred VMs per host. This talk will describe the scalability challenges that were overcome in Citrix XenServer 6.2 to enable support for up to 500 fully virtualised or 650 paravirtualised VMs per host. These include limits with event channels, blktap, xenstored and consoled. It will also discuss how dom0 CPU utilisation was reduced in order to make a large number of VMs responsive and thus usable, and will present benchmark measurements quantifying these improvements.

View presentation          View session recording

Open Source Citrix Windows PV Drivers

Paul Durrant, Citrix

Citrix has recently spent several months making sure all the key parts of XenServer are open source. Part of this effort made the XenServer Windows Paravirtual (PV) drivers available in source form under a BSD 2 clause license on GitHub. Building these drivers outside of the internal Citrix XenServer build environment was quite hard and the resulting binaries would only run correctly in a XenServer host environment. I have recently spent many weeks modifying the drivers so that they should work on any recent upstream Xen host environment thus making it much easier for anyone outside of Citrix to build and deploy the drivers. I would therefore like to give a brief tour of all the drivers, their source, what each of them does, and how they all interact. I will also discuss plans for posting signed versions of these drivers onto Windows Update for general use by the community.

View presentation          View session recording

xenserver-core: What it is, How it is Built and How to get Involved

Euan Harris, Citrix

XenServer is open source and freely available, but it is packaged as an appliance image which must be installed on dedicated hardware. xenserver-core repackages the core components of XenServer so they can easily be built and installed on a standard Linux distribution. Its main goals are: * to make it easy to download, modify and build XenServer components, or just learn how they work; * to help upstream distributions to include up-to-date XenServer packages; * to provide an environment for experimentation. This talk will explain the motivations behind xenserver-core and how it relates to the open-sourcing of XenServer. For developers, it will cover how to get the code, how to build it and how to contribute back to the project. For packagers, it will explain the project's development and release processes and what an upstream maintainer can expect from it.

View presentation          View session recording

SecureServe: A Multi-level Secure Server Virtualization Platform on Xen

Jason Sonnek, Adventium Labs

Due to the rapid shift toward cloud computing, virtual desktop infrastructure (VDI) and thin client computing, many organizations in the government desire a high assurance, multi-level secure server virtualization platform that is low-cost, open and enterprise ready. In this presentation, Jason Sonnek will present SecureServe, a recently launched effort to develop such a platform by building on the open-source Citrix XenServer. The SecureServe project will draw upon research in a number of areas, including dom0 disaggregation, Xen Security Modules mandatory access controls and static/dynamic attestation. In this presentation, Jason will describe the project objectives and requirements, the project's relation to Citrix XenClient XT and XenServer Windsor, current development status and plans for moving forward.

View presentation          View session recording

Delivering Continuous Deployment of Xen-API at Cloud Scale

John Garbutt, Rackspace

Currently xen-api is really only installed today as part of XenServer. It has traditional enterprise style releases, with controlled upgrades and hotfixes when required. When deploying OpenStack Rackspace, with the help of the OpenStack community, have adopted an approach where any check-in could be deployed, and the system upgraded, from any other checkin from that last release, or earlier in the current release. It would be interesting to see if xen-api could move towards a model. At a minimum having more regular check points where an upgrade would be possible. When running a cloud, a very small amount of control plane downtime is possible, but ideally there should be zero downtime for user's virtual machines. We should explore the ability to only upgrade Xen as a last resort, but still be able to update as much the control and data plane as possible, while keeping VMs alive.

View presentation          View session recording

Multiple Device Emulators for HVM Guests

Paul Durrant, Citrix

Currently Xen only allows a single device emulator to be attached to each HVM guest in a system and, to date, this has been QEMU generally running as a process in the same domain as the toolstack, or in a stub domain. To enable the deployment of virtual GPUs to HVM guests in XenServer, patches were created to allow multiple device emulators to be attached to each HVM guest. QEMU continues to be used to emulate the majority of the devices, but a second process is spawned to handle the virtual GPU. This opens up the possibility of the GPU vendors supplying 'appliance' driver domains in future. I'd like to give an overview of the changes that we've made to Xen and QEMU to enable the use of multiple emulators, the potential benefits to driver domains, plus the knock on effect of emulator disaggregation on the 'unplug' protocol and what we could do about this.

View presentation          View session recording

Xen and XenServer Storage Performance

Felipe Franciosi, Citrix

The development of low latency storage media such as modern Solid State Drives (SSD) brings new challenges to virtualisation platforms. For the first time, we are witnessing storage back ends which are so fast that the CPU time spent in processing data significantly impacts the delivered throughput. This is aggravated by CPU speeds remaining largely constant while storage solutions get faster by orders of magnitude. To meet user demands and fully exploit SSD performance under Xen, new technologies are necessary. This talk will discuss the Xen storage virtualisation data path when using various back ends (e.g. blkback, tapdisk, qemu). It will explain why it is hard to exploit SSD performance with current technologies and present measurement data for a variety of workloads. Finally, it will show how techniques such as persistent grants and indirect I/O can help to mitigate the problem.


View presentation          View session recording

Recent comment in this post
Tobias Kreidl
Very interesting information, so thank you , Tim, for posting this. Above all, it is encouraging to see Citrix grabbing the stora... Read More
Monday, 30 December 2013 19:15
Continue reading
38023 Hits
1 Comment

XenServer Status - October 2013

This past month saw some significant progress toward our objective of converting XenServer from a closed source product developed within Citrix to an open source project.  This is a process which is considerably more difficult and detailed than simply announcing that we’re now open source, and I’m pleased to announce that in October we completed the publication of all sources making up XenServer.  While there is considerable work left to be done, not only can interested parties view all of the code, but we have also posted nightly snapshots of the last thirty builds from trunk.  For organizations looking to integrate with XenServer, these builds represent an ideal early access program from which to test integrations.

In addition to the code progress, we’ve also been busy building capabilities and supporting a vibrant ecosystem.  Some of the highlights include:

Now no status report would be complete without some metrics, and we’ve got some pretty decent stats as well.  Unique visitors to in October were up 12% to over 26,000.  Downloads of the core XenServer installation ISO directly from were up by over 1000 downloads.  Mailing list activity was up 50% and we had over 80 commits to the XenServer repositories.  What’s even more impressive with these numbers is that XenServer is built from a number of other open source projects, so the real activity level within XenServer is considerably larger.

At the end of the day this is one month, but it is a turning point.  I’ve been associated in one form or another with XenServer since 2008, and even way back then there were many who expected XenServer was unlikely to be around for long.  Five years later there are more competing solutions, but the future for XenServer is as solid as ever.  We’re working through some of the technical issues which have artificially limited XenServer in recent years, but we are making significant progress.  If you are looking for a solid, high performance, open source virtualization platform; then XenServer needs to be on your list.  If you are looking to contain the costs of delivering virtualized infrastructure, the same holds true.  

More important than all these excellent steps forward is how XenServer can benefit the ecosystem of vendors and fellow open source projects which are required to fully deliver virtualized infrastructure at large scale.  Over the next several months I’m going to be reaching out to various constituencies to see what we should be doing to make participating in the ecosystem more valuable.  If you want to be included in that process, please let me know.

Continue reading
14807 Hits

How did we increase VM density in XenServer 6.2? (part 2)

In a previous article, I described how dom0 event channels can cause a hard limitation on VM density scalability.

Event channels were just one hard limit the XenServer engineering team needed to overcome to allow XenServer 6.2 to support up to 500 Windows VMs or 650 Linux VMs on a single host.

In my talk at the 2013 Xen Developer Summit towards the end of October, I spoke about a further six hard limits and some soft limits that we overcame along the way to achieving this goal. This blog article summarises that journey.

Firstly, I'll explain what I mean by hard and soft VM density limits. A hard limit is where you can run a certain number of VMs without any trouble, but you are unable to run one more. Hard limits arise when there is some finite, unsharable resource that each VM consumes a bit of. On the other hand, a soft limit is where performance degrades with every additional VM you have running; there will be a point at which it's impractical to run more than a certain number of VMs because they will be unusable in some sense. Soft limits arise when there is a shared resource that all VMs must compete for, such as CPU time.

Here is a run-down of all seven hard limits, how we mitigated them in XenServer 6.2, and how we might be able to push them even further back in future:

  1. dom0 event channels

    • Cause of limitation: XenServer uses a 32-bit dom0. This means a maximum of 1,024 dom0 event channels.
    • Mitigation for XenServer 6.2: We made a special case for dom0 to allow it up to 4,096 dom0 event channels.
    • Mitigation for future: Adopt David Vrabel's proposed change to the Xen ABI to provide unlimited event channels.
  2. blktap2 device minor numbers

    • Cause of limitation: blktap2 only supports up to 1,024 minor numbers, caused by #define MAX_BLKTAP_DEVICE in blktap.h.
    • Mitigation for XenServer 6.2: We doubled that constant to allow up to 2,048 devices.
    • Mitigation for future: Move away from blktap2 altogether?
  3. aio requests in dom0

    • Cause of limitation: Each blktap2 instance creates an asynchronous I/O context for receiving 402 events; the default system-wide number of aio requests (fs.aio-max-nr) was 444,416 in XenServer 6.1.
    • Mitigation for XenServer 6.2: We set fs.aio-max-nr to 1,048,576.
    • Mitigation for future: Increase this parameter yet further. It's not clear whether there's a ceiling, but it looks like this would be okay.
  4. dom0 grant references

    • Cause of limitation: Windows VMs used receive-side copy (RSC) by default in XenServer 6.1. In netbk_p1_setup, netback allocates 22 grant-table entries per virtual interface for RSC. But dom0 only had a total of 8,192 grant-table entries in XenServer 6.1.
    • Mitigation for XenServer 6.2: We could have increased the size of the grant-table, but for other reasons RSC is no longer the default for Windows VMs in XenServer 6.2, so this limitation no longer applies.
    • Mitigation for future: Continue to leave RSC disabled by default.
  5. Connections to xenstored

    • Cause of limitation: xenstored uses select(2), which can only listen on up to 1,024 file descriptors; qemu opens 3 file descriptors to xenstored.
    • Mitigation for XenServer 6.2: We made two qemu watches share a connection.
    • Mitigation for future: We could modify xenstored to accept more connections, but in the future we expect to be using upstream qemu, which doesn't connect to xenstored, so it's unlikely that xenstored will run out of connections.
  6. Connections to consoled

    • Cause of limitation: Similarly, consoled uses select(2), and each PV domain opens 3 file descriptors to consoled.
    • Mitigation for XenServer 6.2: We use poll(2) rather than select(2). This has no such limitation.
    • Mitigation for future: Continue to use poll(2).
  7. dom0 low memory

    • Cause of limitation: Each running VM eats about 1 MB of dom0 low memory.
    • Mitigation for future: Using a 64-bit dom0 would remove this limit.

Summary of limits

Okay, so what does this all mean in terms of how many VMs you can run on a host? Well, since some of the limits concern your VM configuration, it depends on the type of VM you have in mind.

Let's take the example of Windows VMs with PV drivers, each with 1 vCPU, 3 disks and 1 network interface. Here are the number of those VMs you'd have to run on a host in order to hit each limitation:

Limitation XS 6.1 XS 6.2 Future
dom0 event channels 150 570 no limit
blktap minor numbers 341 682 no limit
aio requests 368 869 no limit
dom0 grant references 372 no limit no limit
xenstored connections 333 500 no limit
consoled connections no limit no limit no limit
dom0 low memory 650 650 no limit

The first limit you'd arrive at in each release is highlighted. So the overall limit is event channels in XenServer 6.1, limiting us to 150 of these VMs. In XenServer 6.2, it's the number of xenstore connections that limits us to 500 VMs per host. In the future, none of these limits will hit us, but there will surely be an eighth limit when running many more than 500 VMs on a host.

What about Linux guests? Here's where we stand for paravirtualised Linux VMs each with 1 vCPU, 1 disk and 1 network interface:

Limitation XS 6.1 XS 6.2 Future
dom0 event channels 225 1000 no limit
blktap minor numbers 1024 2048 no limit
aio requests 368 869 no limit
dom0 grant references no limit no limit no limit
xenstored connections no limit no limit no limit
consoled connections 341 no limit no limit
dom0 low memory 650 650 no limit

This explains why the supported limit for Linux guests can be as high as 650 in XenServer 6.2. Again, in the future, we'll likely be limited by something else above 650 VMs.

What about the soft limits?

After having pushed the hard limits such a long way out, we then needed to turn our attention towards ensuring that there weren't any soft limits that would make it infeasible to run a large number of VMs in practice.

Felipe Franciosi has already described how qemu's utilisation of dom0 CPUs can be reduced by avoiding the emulation of unneeded virtual devices. The other major change in XenServer 6.2 to reduce dom0 load was to reduce the amount of xenstore traffic. This was achieved by replacing code that polled xenstore with code that registers watches on xenstore and by removing some spurious xenstore accesses from the Windows guest agent.

These things combine to keep dom0 CPU load down to a very low level. This means that VMs can remain healthy and responsive, even when running a very large number of VMs.

Recent comment in this post
Tobias Kreidl
We see xenstored eat anywhere from 30 to 70% of a CPU with something like 80 VMs running under XenServer 6.1. When major updates t... Read More
Wednesday, 13 November 2013 17:10
Continue reading
27607 Hits
1 Comment

Xen Project User Summit Videos are Available

For those community members, who could not attend the Xen Project User Summit in New Orleans, we now published all the videos on the Xen Project's video stream. The summit program included quite a few XenServer sessions, which you may want to check out.

For your convenience, we also created a portal page on that links to all recordings of the user summit. Again, a big Thank You to our speakers and to Russell Pavlicek for organizing the event.

Continue reading
7366 Hits

Xen Project Developer Summit, Oct 24-25, Edinburgh


If you are using XenServer, then you are using software developed by the Linux Foundation Xen Project. More specifically you will be using the Xen Hypervisor and XAPI. If you want to know about the latest trends, cutting edge features and innovations that are developed by the Xen Project and will be coming to Citrix products in future, the Xen Project Developer Summit may be for you!

The Xen Project Developer Summit is the Xen Project’s annual developer conference. It brings together the developers and power users that define the Xen Project. We will be sharing ideas and experiences, discuss the latest technical developments and innovations and plan the evolution of Xen and its sub projects for 2014.

This year's summit features sessions covering the latest developments and innovations in the Xen Hypervisor, Client Virtualization, Security, Graphics and Audio Virtualization and Cloud Computing. With the addition of ARM support to the Xen Hypervisor, we will also show and discuss new use-cases for virtualization on mobile devices, in automotive and middlebox appliances such as firewalls and NATs.

The summit also features many XenServer related sessions, such as:

For more information, check out the event schedule. You can register for the event here. Spaces are limited.

Continue reading
7917 Hits

VM Density and Project Pulsar

While overcoming the architectural obstacles in order to allow XenServer 6.2.0 to run up to 500 HVM guests per host, we came across an interesting observation: even for apparently idle Windows 7 VMs, there is some amount of work done in dom0. With careful analysis and optimisations in the way we emulate virtual hardware for such VMs, we managed to eliminate most of this work for guests that are idle. This allows us to effectively run 500 Windows VMs and keep dom0 load average practically at zero (after the VMs have finished booting).

The busy bees

We started by observing the CPU utilisation of the QEMU process that is responsible for the hardware emulation of a Windows 7 guest. Even when the guest was just waiting for login and not running disk indexing services, Windows Update or other known I/O operations, the QEMU process would consistently consume between 1% and 3% of CPU time in dom0 (depending mostly on the host's characteristics). In this scenario, there is one QEMU process for each HVM guest running on a host.

When we succeeded at overcoming the hard limits that prevented us from starting 500 guests (see this post), we soon realised that the small amount of work done by each QEMU would quickly add up. With 3% of dom0 vCPU consumption per running Windows 7 VM, it takes just over 130 VMs to completely max out four dom0 vCPUs. At that point, dom0's load average quickly increases and the host's performance becomes compromised.

Without available CPU time, dom0 is unable to promptly serve network and storage I/O requests. Other control plane operations are also affected since the toolstack (e.g. xapi, xenopsd, xenstore) will struggle to run and respond to requests from XenCenter or other clients.

Project Pulsar

In order to investigate why those QEMUs were "spinning" so much, we created Project Pulsar (named after neutron stars that spin considerably). The project conducted a detailed analysis of every event that disturbed QEMU from its otherwise sleeping state.

With careful debugging, we learned that there are two categories of events that cause QEMU to wake up. Firstly, there are internal timers that occur several times a second to poll for certain events (e.g. buffered I/O). Secondly, there are actual interrupts generated by guests to check on certain virtual hardware (e.g. USB and the parallel port).

The following table shows the amount of events disturbing QEMU during the lifetime of a Windows 7 VM running for 5 minutes. The events are grouped in 30 seconds time intervals for simplicity. This facilitates the visualisation of the boot period (within the first 30 seconds interval), the time that the VM is allegedly idle and the shutdown phase at the end. We separated the columns into Timer, Read and Write Events. Timer events are internal to QEMU and Read or Write events come from the VM as interrupts. After the PV drivers are loaded (which happens during the boot), storage and network I/O are not handled by QEMU and therefore are not accounted anymore.

30 Secs Interval Timer Events Read Events Write Events
000 2,130 160,191 59,769
001 3,762 17,831 1,731
002 3,603 10,420 1,662
003 3,458 2,964 1,587
004 3,451 2,970 1,591
005 3,454 3,038 1,650
006 3,461 3,078 1,683
007 3,457 2,964 1,587
008 3,429 2,952 1,581
009 3,446 3,034 1,657
010 594 347 280

While certain events only happen during the guest initialisation or shortly after the boot completed (e.g. parallel port scans), others remain constant throughout the life of the VM (e.g. USB protocols).

Idling down dom0

To address the first category of events that disturb QEMU (e.g. internal timers), we first studied why they were ever required. As it turned out, newer versions of QEMU were already patched to disable some of these. A good example is the case of buffered I/O which required polling. With the creation of a dedicated event channel for buffered I/O interrupts (see this patch) and a corresponding change to the device model (see this patch), QEMU no longer needs to poll.

The other timers we identified are necessary only when certain features are enabled. These are the QEMU monitor (that allows debugging tools to be hooked up to QEMU processes), serial port emulation and VNC connections. Considering XenServer does not support QEMU debugging via its monitor feature nor direct serial connections to HVM guests, these could be safely disabled. The last timer would only be active when a VNC connection is established to a guest (e.g. via XenCenter).

The second category of events happens due to the very nature of the hardware that is being emulated. If we present a Windows VM with an IDE DVD-ROM drive, the guest OS handles this (virtual) drive as a real drive. For example, it will poll the drive every so often to check whether the media has changed. Similar types of interrupts will be initiated by the guest to communicate with other emulated hardware.

In order to address these, we modified Xapi to allow the emulation of USB, parallel and serial ports to be turned off. Parallel port emulation fits in the same category as the serial port (i.e. there is no supported way to plug virtual devices to these ports in a guest) and the emulation of both are fairly safe and symptomless to be turned off.

Disabling USB, however, may have side effects. When using VNC to connect to a guest's console, a USB tablet device driver is used to allow for absolute coordinates of the mouse on the screen. When not using this USB driver, the VNC falls back to a PS/2 emulation which can only provide relative mouse positioning. The side effect is that, without USB, the mouse pointer of the VNC client will very likely be misaligned with the mouse pointer in the guest. This makes the console very hard to use.

The good news is that the Windows Remote Desktop Protocol (RDP) does not rely on the USB tablet driver. If the guest is configured to allow RDP connections, the USB emulation can be disabled without this side effect. When available, XenCenter already prefers RDP over VNC connections to Windows VMs by default.

Configuring the toolstack

The recommendations of Project Pulsar were adopted as defaults wherever possible and have been incorporated in XenServer 6.2.0. These included changes not visible to the VM (such as QEMU internal timers). However, we decided not to change the virtual hardware presented to VMs unless this is explicitly configured.

In order to configure Xapi to disable the Serial Port emulation, use the following command:

xe vm-param-set uuid=<vm-uuid> platform:hvm_serial=none

Similarly, the Parallel Port emulation can be disabled as follows:

xe vm-param-set uuid=<vm-uuid> platform:parallel=none

Finally, the USB emulation can be disabled as follows:

xe vm-param-set uuid=<vm-uuid> platform:usb=false
xe vm-param-set uuid=<vm-uuid> platform:usb_tablet=false

Note that two commands are necessary to completely disable the USB emulation. These disable both the virtual USB hub and the virtual tablet device used for positioning the mouse.

The best way to disable the emulation of the DVD-ROM drive is to delete the associated VBD. For information on how to do that, refer to Section A.4.23.3 of the XenServer 6.2.0 Administrator's Guide.


The following table shows the amount of events disturbing QEMU during the lifetime of a Windows 7 VM running for 5 minutes. This is the same VM used for the numbers in the table above, except that this time we incorporated the timer patches in QEMU and disabled the USB, DVD-ROM, Monitor, Parallel- and Serial- port emulation.

30 Secs Interval Timer Events Read Events Write Events
000 3 146,114 57,254
001 0 0 1
002 0 0 0
003 0 0 0
004 0 0 1
005 0 0 0
006 0 0 0
007 0 0 0
008 0 0 0
009 0 8 13
010 0 58 108

With these figures, we are able to start 500 Windows 7 VMs on one host and keep dom0 load average practically at zero (after the VMs have booted). 

Recent Comments
Jonathan Pitre
Could you please provide a bash script to disable the serial and parallel port for all VMs ? something similar to the scripts we c... Read More
Monday, 11 November 2013 23:36
Mark Starikov
something like that maybe? for vms in $(xe vm-list is-control-domain=false is-a-template=false is-a-snapshot=false | grep uuid | a... Read More
Wednesday, 20 November 2013 21:51
srinivas j
@Felipe, thanks for the blog post ! are the QEMU timer patches inside XenServer 6.2? or is there a plan? All the XenServer custome... Read More
Wednesday, 01 January 2014 12:42
Continue reading
21086 Hits

Building xenserver-core

In a previous post, Dave Scott showed how easy it now is to install the core components of XenServer on a standard installation of CentOS, using xenserver-core. Since then, we have been hard at work adding features and functionality, and making it even smoother and easier to install from our repositories. However, since developers are a major part of the audience for xenserver-core, we have also paid a lot of attention to making it as smooth and easy as possible to build the packages for yourself.


On RPM-based distributions, the packages are built using a tool called mock.   Mock builds each new package in a fresh chroot environment, ensuring that your system isn't affected by the by-products of building packages, and that missing dependencies in the package specifications don't go unnoticed.  To install it on a RHEL/CentOS system then you will need to add the EPEL repositories. Here is a useful article for CentOS.

After adding EPEL, install and set up mock:

yum install -y mock rpm-build

Mock will refuse to run as root - you can run it using your own user account, or a special account you create for this purpose. To add this account to the mock group, type the following as root:

useradd  -G mock <username>
passwd <username>

su - <username>

Building the packages

With the prerequisites installed, building xenserver-core is a 4-step process:

  1. Clone the xenserver-core repository.   If you are planning to make changes, you should fork the repository on GitHub and clone that instead.   To clone the main xenserver-core repository, type:

    git clone
  2. This configures mock and initializes an RPM repository in the RPMS directory, where the built packages will be stored.   Enter the directory created by git (by default, it will be called "xenserver-core") and type:

  3. Generate the makefile which will actually build the xenserver-core packages:

     ./ > Makefile
  4. Start the build:


Building all the xenserver-core packages takes around 90 minutes on a reasonably fast machine, and requires a couple of gigabytes of free disk space.   When the build finishes, you will find source packages in the SRPMS directory, and binaries in RPMS.

Installing your new packages

The most flexible way to install your packages is to copy newly-built RPM repository to a web server and configure YUM on your target machine to download packages from there.  Assuming that you already have a web server set up,  copy the RPMS and SRPMS directories to your server's DOCROOT directory.

To configure the target machine to use your new repository, create /etc/yum.repos.d/xapi.conf, filling in your repository's baseurl as appropriate:

name=CentOS-$releasever - xenserver-core

name=CentOS-$releasever - xenserver-core

After this, type yum install xenserver-core to install the packages from your new repository. Once the installation is complete, run the xenserver-install-wizard command to configure xenserver-core.   Please note that, unless you were already running under Xen, you will have to reboot to complete the installation.

What's going on here?

When you cloned the xenserver-core repository you may have noticed that it doesn't actually contain any software.   Instead, it has a collection of RPM SPEC files, along with the build scripts we ran above.   When you run make, it downloads each package's source code from the upstream repository, builds a source RPM, then uses mock to compile the binary RPM, as you can see in this snippet of build output:

[CURL] SOURCES/libvhd-0.9.1.tar.gz
[RPMBUILD] SRPMS/ocaml-libvhd-0.9.1-1.src.rpm
[MOCK] RPMS/x86_64/ocaml-libvhd-0.9.1-1.x86_64.rpm
[CREATEREPO] RPMS/x86_64/ocaml-libvhd-0.9.1-1.x86_64.rpm

Please give us feedback

We want to make xenserver-core as easy as possible to install, build and use. We would love to hear what you think of our progress so far.   You can contact us through comments on this blog.  If you have development questions about XenServer and its components, try the This email address is being protected from spambots. You need JavaScript enabled to view it. mailing list.

If you run into any problems or bugs in xenserver-core's packaging or build scripts, please raise an issue on our GitHub issues page.

Prefer Ubuntu?

Following the instructions above will build xenserver-core for an RPM-based distribution such as Fedora or CentOS.   What if you prefer Ubuntu or another Debian-based distribution?   Try cloning the repository onto your Ubuntu machine and running the build there.   The results may surprise you!

More updates coming soon...

Continue reading
24497 Hits

Introducing Open Source XenRT

As a follow up activity to the open sourcing of XenServer, Citrix is pleased to announce the open sourcing of its automated test platform, XenRT.

XenRT ("Xen Regression Test") is a test automation framework, written in Python, providing abstractions for the various components under test (pool, host, VM, storage, network etc). The library code which makes up these abstractions simplifies the process of writing tests, allowing quite complex operations to be performed in a single method call.

In a full deployment, XenRT handles all aspects of the testing process - it will schedule a test job onto a host, bootstrap it (via DHCP/PXE), install the build to be tested, carry out the testing, and collect all necessary logs for troubleshooting, without any user interaction required.

In addition to basic functional, regression, and stress testing, XenRT has suites of tests that are used for testing performance, scalability, and interoperability.

Within Citrix, XenRT is used with a distributed lab comprised of an extremely wide range of hardware, and is developed and maintained by a team of some 25 developers. Tests are also written and executed directly by the wider XenServer engineering team, in a true "Test-as-a-Service" platform - see this post on the Citrix blog for more information.

XenRT has been open sourced to leverage Citrix's experience and resources in test automation to help improve the quality of open source Xen and XenServer releases, to benefit the entire community.

To get started with XenRT, follow the links below to the code and a README document (which contains getting started instructions - further documentation will follow in the near future). For discussion a mailing list has been created - information about this can be found at

Download links:
README document
Main XenRT tarball
Third party test resource tarball
Source for third party resources (not required for normal operation)

Continue reading
17799 Hits
1 Comment

The XenServer Project Plan and Roadmaps

Citrix XenServer is undergoing a transition to an open and transparent development model.  This model will enable users, customers, partners, vendors, consultants and pretty much anyone with an interest in the future and direction of XenServer become active participants.  The transition from a closed development model to an open one doesn’t take place overnight, and it doesn’t come without its share of challenges.  As we work towards completing this transition and fulfilling the promise of a vibrant open source project, this page will contain the current status of our efforts, and the next major milestones. 

In traditional product management there is always discussion about “roadmaps”.  Since we’re transitioning from closed development practices there will undoubtedly be those who wish to see a roadmap for XenServer.  I hate to disappoint, but we don’t have one.  Instead we have a development and project plan.  The plan shows the current prioritization of feature development, and the current development status.  Work is actively underway to make that public, but it will take a bit more time.  Of course, that then leads people to conclude that an underlying roadmap with specific feature deliveries forms part of the plan.  While true in theory, in open source development things are just a little bit different.  With an open source project, it’s not the features that matter but rather support of those features.

Long time users of XenServer will be very familiar with the “Supported”, “Experimental” and “Unsupported” labels given to features, and that model will continue as we move forward.  In fact what distinguishes a release is the date of supportability and the support labels given to features.  For those unfamiliar with these labels, and to refresh for those who are, here is how we define them for those people who have software maintenance with Citrix:

  • Supported – Customers will receive support from Citrix for their implementation of the feature
  • Experimental – This label tends to be applied to implementations of a feature which function within a narrow scope.  We’ve used it multiple times over the years and in practice this means that Citrix will provide support on a “best effort basis”, but that at the end of the day the code is experimental and it might not be possible to do what the customer expected.
  • Unsupported – Customers are on their own if they have made modifications to the system which are outside the expected configuration parameters.  This was and remains an important item as we have a general purpose Linux distribution (dom0) as part of the shipping product and as such Linux experts can and do install packages or make changes which could render the system inoperable.  Often customers enter into this territory with the best of intentions and our escalation engineers do try to provide some level of support as part of goodwill.  I would expect that anyone building their own XenServer from source would be 100% unsupportable as we’d have no way to know what they’ve done.

In practice this means that until something has official support, the support burden will be borne by the community.  Once a release occurs, then certain features will potentially become supported.  So what community members can expect is that when we release we will clearly state which features are Supported or Experimental and anything which has no specification can be assumed Unsupported with the support burden remaining with the community itself.  Further with each release will also come a date when support or maintenance won’t be available.  As with all shipping and supported Citrix products, this is listed in the product lifecycle matrix.

So with that as the backdrop, here are the next items on our agenda:

Major project activity: Publish project plan through next release cycle

Next release cycle: Q1 2014

Recent comment in this post
Tobias Kreidl
Hi, Tim: Thanks for this article. The open-sourcing of Xenserver has indeed generated a lot of positive buzz. One thought concerns... Read More
Friday, 23 August 2013 03:08
Continue reading
12669 Hits
1 Comment

The Xen Project announces User Summit line-up!

The Xen Project has announced the sessions for the 2013 Xen Project User Summit to be held in conjunction with Linuxcon North America next month in New Orleans. This is the first time that the the Xen hypervisor team will be having a conference focused entirely on the users of the Xen hypervisor. The Xen Project develops the Xen Hypervisor and XAPI toolstack, both core components of XenServer.


Keynote Address: Xen: This is not your Dad’s hypervisor!

Jay Williams, VP of Product Management for CA Technologies, will deliver the keynote address.  He will explain why Xen's strengths are critical for powering CA AppLogic and platforms like OpenStack.

Featured Talk: Free yourself from the tyranny of your cloud provider!

Greg Kroah-Hartman, maintainer of the stable branch of the Linux kernel (among a mass of other things), will discuss how using kexec in a paravirtualized user domain, with no changes to the control Domain or Xen itself, can allow you to boot your own kernel, no matter what the hosting provider is forcing you to run.

And a whole lot more...

Attendees will also learn about:

  • how Xen works in the cloud with Apache CloudStack and OpenStack
  • the effort to bring Xen to CentOS 6
  • the independent project to create a GUI for XAPI via Xen Orchestra
  • how Xen can empower new ARM-based servers
  • using the Ceph filesystem with Xen under CloudStack
  • a case study involving Xen and security auditing for Navis Lighthouse

But what if I am relatively new to Xen?

Folks new to Xen will benefit from an introductory session, as well as a short session to explain the differences between Xen Project, XenServer, and XAPI.  Everyone, from long-time Xen users to those just kicking the tires, will receive plenty of useful information.

The talks currently scheduled include:

  • Xen: This is not your Dad’s hypervisor! by Jay Williams of CA Technologies
  • Xen for Beginners by Bryan Smith of Tacit Labs, Inc.
  • Xen, XenServer, and XAPI: What’s the Difference? by James Bulpin and Russell Pavlicek of Citrix Systems
  • ARM Servers and Xen – Hypervisor support at Hyperscale by Larry Wikelius of Calxeda
  • Bringing Xen Dom0 to CentOS-6 by Karanbir Singh of the CentOS Project
  • Free yourself from the tyranny of your cloud provider! by Greg Kroah-Hartman of the Linux Foundation
  • Xen Orchestra (XAPI and XenServer from the web) by Olivier Lambert of Vates
  • Network Multi-tenancy in Xen-based Clouds by Chiradeep Vittal of Citrix Systems
  • Building the Rackspace Open Cloud with XenServer and OpenStack by John Garbutt of Rackspace
  • Coalfire Systems' Navis Lighthouse Gen 2 - Security Audit Appliance by Grant McWilliams of Coalfire Systems
  • Ceph, Xen, and CloudStack: Semper Melior by Patrick McGarry of Inktank

The schedule is subject to change due to speaker availability.  As the User Summit is co-located with LinuxCon North America and CloudOpen North America, you can find the full schedule with times and descriptions on the LinuxCon NA Schedule page.We hope to see you in New Orleans in September!

More Information

Continue reading
8654 Hits

Making sense of XenServer vs. xenserver-core vs. Citrix XenServer

So XenServer is now open-source, what does that mean? I look at XenServer as two things: firstly a set of components selected and engineered to work together as a system; and secondly a Linux distribution to provide the base platform to host and execute those components. Of course these two things are tightly coupled because the choice of base Linux distro and the set of packages installed will be part of the story when engineering the system as a whole. However we want to make it such that XenServer's core components, the stuff that does all the virtualization, management, monitoring and so on, can be used on a variety of Linux distros. This means we need to cleanly separate the components from the base distro, e.g. making XenServer components work with any reasonable distro and avoiding making assumptions about particular versions etc.

Let's start with the core components and refer to them collectively as "xenserver-core" (e.g. that could be the name of the meta-package to install them all to a distro "yum install xenserver-core"/"apt-get install xenserver-core" as used in Dave Scott's recent tech preview). These components include the xapi tool stack, storage manager, network daemon and related tools, HA daemon, etc. A second group of core components includes Xen, the Linux kernel, libvirt and qemu; although considered core components it is desirable to be able to use existing distro versions where possible. With suitable package dependencies it should be possible to manage all of the above.


It's important to remember that many of the core components are derived from upstream projects. For example the xapi tool stack is part of the Linux Foundation's Xen Project but is consumed by XenServer, you could think of the XenServer version of xapi being a short term fork of the upstream code. In practice I don't expect to see much divergence between XenServer's xapi and the upstream xapi because it's the same people working on both and XenServer is the primary consumer of the project. For other components that are more widely used, such as qemu, libvirt and Xen I would expect short term divergence as critical features and fixes are ported to the version used by XenServer (just like Linux distros do) but with a rule that all required code is upstreamed to the relevant project to avoid long term divergence.

OK, we have xenserver-core which can now be installed on top of any reasonable Linux distribution. So when I talk about "XenServer" what do I mean? In general I mean the end result of both aspects of XenServer, the components and the base distro, all wrapped up in an ISO with an installer of some kind. This "appliance" model is how XenServer has been for years and provides a turn-key virtualization platform that does not require Linux sysadmin experience to install. This means we start with xenserver-core packages, choose a particular base distribution and set of packages from it and glue the whole lot together somehow. In a sense this is a distro-customization exercise. XenServer's build system has been doing this since day one albeit in a rather more complex way than described above. As part of the open-sourcing of XenServer we need to clean up this packaging and assembly phase by using standard tools and methods to take the core components and an off-the-shelf Linux distro and put the two together. This tooling, and all the configuration management (which versions of which packages etc.) will become part of the project.


What does Citrix actually release? Citrix XenServer is a particular instance of the XenServer appliance built, packaged, assembled, tested, warranted and certified by Citrix. It is only Citrix XenServer that can be supported by Citrix (remember that Citrix XenServer is free, anyone can use it without paying, but to get support and maintenance a package can be purchased from Citrix).

If you want XenServer you have some choices of how to get it (once all the necessary pieces are published of course):

  1. xenserver-core installed on a distro of your choice (either compile the components yourself or use a distro or binary release)
  2. XenServer appliance - you can assemble one of these from core components and a base Linux distro using the tools on
  3. Citrix XenServer - like option 2 above but let Citrix do the hard work of build and assembly and benefit from the system testing and certification this binary build will get. This also gives you the option to buy support from Citrix.
Recent Comments
James, You wrote that "we have xenserver-core which can now be installed on top of any reasonable Linux distribution." Although... Read More
Monday, 22 July 2013 21:45
James Bulpin
GizmoChicken - hopefully there will be packages available for CentOS 6.4, Debian and Ubuntu at some point however there is still q... Read More
Wednesday, 07 August 2013 17:55
Matthew Fusaro
So does this mean that, within reason, we can essential build an appliance using other version of Xen? 4.3 for example? Im sure th... Read More
Saturday, 03 August 2013 20:58
Continue reading
33573 Hits

How did we increase VM density in XenServer 6.2?

One of the most noteworthy improvements in XenServer 6.2 is the support for a significantly increased number of VMs running on a host: now up to 500 Windows VMs or 650 Linux VMs.

We needed to remove several obstacles in order to achieve this huge step up. Perhaps the most important of the technical changes that led to this was to increase in the number of event channels available to dom0 (the control domain) from 1024 to 4096. This blog post is an attempt to shed some light on what these event channels are, and why they play a key role in VM density limits.

What is an event channel?

It's a channel for communications between a pair of VMs. An event channel is typically used by one VM to notify another VM about something. For example, a VM's paravirtualised disk driver would use an event channel to notify dom0 of the presence of newly written data in a region of memory shared with dom0.

Here are the various things that a VM requires an event channel for:

  • one per virtual disk;
  • one per virtual network interface;
  • one for communications with xenstore;
  • for HVM guests, one per virtual CPU (rising to two in XenServer 6.2); and
  • for PV guests; one to communicate with the console daemon.

Therefore VMs will typically require at least four dom0 event channels depending on the configuration of the VM. Requiring more than ten is not an uncommon configuration.

Why can event channels cause scalability problems when trying to run lots of VMs?

The total number of event channels any domain can use is part of a shared structure in the interface between a paravirtualised VM and the hypervisor; it is fixed at 1024 for 32-bit domains such as XenServer's dom0. Moreover, there are normally around 50--100 event channels used for other purposes, such as physical interrupts. This is normally related to the number of physical devices you have in your host. This overhead means that in practice there might be not too many more than 900--950 event channels available for VM use. So the number of available event channels becomes a limited resource that can cause you to experience a hard limit on the number of VMs you can run on a host.

To take an example: Before XenServer 6.2, if each of your VMs requires 6 dom0 event channels (e.g. an HVM guest with 3 virtual disks, 1 virtual network interface and 1 virtual CPU) then you'll probably find yourself running out of dom0 event channels if you go much over 150 VMs.

In XenServer 6.2, we have made a special case for our dom0 to allow it to behave differently to other 32-bit domains to allow it to use up to four times the normal number of event channels. Hence there are now a total of 4096 event channels available.

So, on XenServer 6.2 in the same scenario as the example above, even though each VM of this type would now use 7 dom0 event channels, the increased total number of dom0 event channels means you'd have to run over 570 of them before running out.

What happens when I run out of event channels?

On VM startup, the XenServer toolstack will try to plumb all the event channels through from dom0 to the nascent VM. If there are no spare slots, the connection will fail. The exact failure mode depends on which subsystem the event channel was intended for use in, but you may see error messages like these when the toolstack tries to connect up the next event channel after having run out:

error 28 mapping ring-refs and evtchn
message: xenopsd internal error: Device.Ioemu_failed("qemu-dm exited unexpectedly")

In other words, it's not pretty. The VM either won't boot or will run with reduced functionality.

That sounds scary. How can I tell whether there's sufficient spare event channels to start another VM?

XenServer has a utility called "lsevtchn" that allows you to inspect the event channel plumbing.

In dom0, run the following command to see what event channels are connected to a particular domain.


For example, here is the output from a PV domain with domid 36:

[root@xs62 ~]# /usr/lib/xen/bin/lsevtchn 36
   1: VCPU 0: Interdomain (Connected) - Remote Domain 0, Port 51
   2: VCPU 0: Interdomain (Connected) - Remote Domain 0, Port 52
   3: VCPU 0: Virtual IRQ 0
   4: VCPU 0: IPI
   5: VCPU 0: IPI
   6: VCPU 0: Virtual IRQ 1
   7: VCPU 0: IPI
   8: VCPU 0: Interdomain (Connected) - Remote Domain 0, Port 55
   9: VCPU 0: Interdomain (Connected) - Remote Domain 0, Port 53
  10: VCPU 0: Interdomain (Connected) - Remote Domain 0, Port 54
  11: VCPU 0: Interdomain (Connected) - Remote Domain 0, Port 56

You can see that six of this VM's event channels are connected to dom0.

But the domain we are most interested in is dom0. The total number of event channels connected to dom0 can be determined by running

/usr/lib/xen/bin/lsevtchn 0 | wc -l

Before XenServer 6.2, if that number is close to 1024 then your host is on the verge of not being able to run an additional VM. On XenServer 6.2, the number to watch out for is 4096. However, before you'd be able to get enough VMs up and running to approach that limit, there are various other things you might run into depending on configuration and workload. Watch out for further blog posts describing how we have cleared more of these hurdles in XenServer 6.2.

Continue reading
69903 Hits

Tech preview of XenServer + libvirt + ceph

One of the benefits of making XenServer fully open-source is that it’s easier to integrate directly with other great open-source projects. A project that I’m particularly interested in is Ceph: a distributed storage system which is particularly suitable for storing VM disk images in the cloud. The Ceph community has already integrated support directly into two other open-source projects:

  • qemu: the virtual hardware emulator used by Xen and KVM
  • libvirt: an open-source virtualisation toolkit 

All that’s needed to make Ceph work with XenServer is therefore:

  1. to use a newer version of qemu (in Xen jargon this is called “the upstream qemu”)
  2. to integrate libvirt with the XenServer toolstack (i.e. xapi, xenopsd and friends)

After much coffee-fuelled late-night hacking^Wsoftware development, I’m pleased to announce the availability of a “technology preview” of XenServer integrated with Ceph via libvirt. To give it a go, first install yourself a normal CentOS 6.4 x86_64 system. Second login as root and type:

rpm -ihv

This will add the experimental preview software repository based on the Xen code already in CentOS. Next type:

yum install xenserver-core

-- notice how easy it is to install the core packages of XenServer using the normal CentOS distro tools. Now that XenServer is fully open-source, expect to see more like this in future!

It’s now necessary to set up a basic XenServer configuration. The easiest way is to use a simple install “wizard” by typing:


After the wizard has done its magic and you’ve rebooted, you should be able to connect XenCenter like you would with a regular XenServer.

Assuming you have already configured a Ceph cluster, you need to configure your machine to be a Ceph client, as you would with a regular CentOS host. I recommend having a read of the excellent Ceph documentation. Once complete you should be able to list the currently available Ceph storage pools:

ceph osd lspools

Next you should create a XenServer “Storage Repository” to allow VM virtual disks to be stored on Ceph-- this is where libvirt comes in. In dom0, create a libvirt “storage pool” XML file, as if you were going to issue a “virsh pool-create”. I created a file “ceph.xml” which looks like the following:

<pool type='rbd'>
    <host name='' port='6789'/>

Then type:

xe sr-create type=libvirt name-label=ceph device-config:xml-filename=ceph.xml

You should now have a functioning XenServer Storage Repository which can be managed via the XenAPI and from XenCenter. At this point you should be able to install VMs (both PV and HVM should work) and run them from the Ceph storage.

For more in-depth info about how it all works, known issues and ways you could get involved, have a look at the:


Recent Comments
Tobias Kreidl
Dave, Awesome... Ceph (or something that can be extended to huge storage farms, supports thin provisioning, block and logical I/O,... Read More
Monday, 08 July 2013 11:03
Neil Levine
Hi Tobias, I'm the VP Product at Inktank, who sponsor the Ceph project. We are working on a GUI for Ceph which will be avaiable as... Read More
Tuesday, 09 July 2013 16:17
Tobias Kreidl
Hello, Neil: Thank you for your note. Will Inktank's ceph GUI be integrated with the general XenServer management interface (not s... Read More
Wednesday, 10 July 2013 05:01
Continue reading
84251 Hits

Evolving XenServer for the cloud

Today I presented at the CloudStack Collaboration Conference 2013 in Santa Clara on evolving XenServer to better meet the needs of large-scale cloud deployments. XenServer started life as "XenEnterprise" in 2006 and was aimed at SMBs and Enterprises and targeted Windows IT pros who may not have Linux experience. Therefore a lot of the Linux structure of XenServer was hidden. With many cloud shops being Linux shops this hiding is a barrier; for example XenServer's non-standard installation model doesn't fit well with large scale server deployment and management tools like Chef and Puppet - XenServer is built on a standard Linux OS so why can't we deploy lit like a standard Linux OS?

I outlined some of the initiatives currently underway within Citrix that will continue under the project:

Hyperspace: fixing up the build and packaging to allow individual components to be easily built without complex build system requirements. Making XenServer be a "Linux distro+packages" where we take an off-the-shelf Linux distro, add the XenServer components as packages, and bundle the whole thing as XenServer.

Fusion (I called it "Project Upstream" in the presentation to avoid conflict with a commercial product name): getting libvirt and upstream qemu into XenServer.

Windsor: making XenServer more modular, exploiting dom0 disaggregation.

As we move more of XenServer project planning and development to I look forward to more discussion here on these initiatives and the wider topic of making XenServer the best platform for the cloud.

You can find a copy of my slides at CCC13_EvolvingXenServerForTheCloud_JamesBulpin_20130625.pdf

A video of the talk should be up on soon.


Recent Comments
Tobias Kreidl
Thank you, first off, for your efforts as well as push in this exciting direction. Aside from architectural inter-operational aspe... Read More
Saturday, 13 July 2013 19:41
Garrett Taylor
How about ceph caching in RAMDISK? Expensive, obviously, but greased lightning for VMs.
Wednesday, 17 July 2013 17:33
Tobias Kreidl
Interesting (and expensive) thought. The ceph architecture should allow incorporating various external cache mechanisms quite read... Read More
Thursday, 18 July 2013 10:53
Continue reading
12564 Hits

Introducing XenServer the Project

b2ap3_thumbnail_220px-Opensource.pngThis is a big day for the XenServer product as version 6.2 is now available, it is now not only a product supported by Citrix but also becomes a fully open source project that invites participation from the user community. Citrix will still provide a XenServer product which includes support for those customers who chose to purchase a support license.  

Why Open Source Xen

Much of XenServer already was open source, leveraging packages from the Xen Project, Linux kernel and the XAPI Project. We believe that open source plays a strategic role in the future of virtualization and cloud technology and that only open source offers the opportunities for collaborative, open innovation and the economies of scale that these markets demand. By open sourcing XenServer, customers, partners and developers gain full public visibility into the ongoing development and future of XenServer and can directly engage with us to contribute new XenServer functionality, build deeper integrations and steer the architectural direction of the platform. In summary, an open source strategy was chosen because: 

Best of Both Worlds Free Software, Commercial Support Available

With the change to open source, XenServer is now available for free to everyone on the new community. Citrix will provide support for a distribution of XenServer. This is very similar to leading open source enterprise products like  Red Hat Enterprise Linux verses say CentOS Linux. By purchasing Citrix XenServer support licenses you get:

  • Citrix Premier 24x7 worldwide support offering unlimited-incident worldwide technical support in nine languages across the globe.
  • Commercially packaged and certified product that goes through rigorous testing and provides certified product lifecycles and guaranteed support statements.
  • Simplified patching and updating via XenCenter management console that enables automated GUI driven patch application and upgrades. In contrast, free XenServer requires patches and updates to be installed from the command line, but both include XenCenter for overall server management.
  • Indemnification and license protection that protects customers from any liabilities stemming from the open source code.
  • Citrix knowledgebase & My Account Portal for access to the expanded inventory of technical articles and a simple-to-use portal for product downloads and license management.

What to Expect from 

So this is just the beginning of We'll begin by releasing the source code and providing binaries for users to download and install. For now we'll provide mailing lists and still use the existing forums provided by Citrix. Over time we'll transition them to this site. Additionally, we'll add all the things that you would expect from an open source project including a way to track bugs, request features and interact with the developers to collaborate on the code. We also include a number of ways to ask questions and get support via the mailing lists, forums and our Q&A system. 

Recent Comments
Xinli Niu
Wow, Great news!
Thursday, 27 June 2013 03:04
Does that mean all features are free now? Read More
Sunday, 30 June 2013 10:25
Mark R. Hinkle
Yes see the Q&A - Read More
Monday, 01 July 2013 02:36
Continue reading
36681 Hits

Citrix XenServer 6.1 Wins the “Best Virtualization Product” Award

[originally posted on the Citrix Blog by Sarah Vallieres]

Citrix XenServer 6.1 has won the “Best Virtualization Product” award from “IT Decision-Making in the Future – 2012 Outstanding Enterprise-Class IT Products Selection”, organized by TechTarget China.

The reason: Citrix XenServer 6.1 is the industry’s leading enterprise-class virtualization platform. XenServer can create and manage virtualization infrastructure of server, desktop and cloud computing, and can provide a shortcut in transitioning to cloud computing for enterprises.

The award is one of TechTarget China’s Products of the Year. The award was jointly organized by TechTarget China’s 14 websites, including storage, network, database, virtualization, cloud computing and data center management. The selection was made by evaluating various aspects of products, including cost-effectiveness, technical service, reliability, manageability, scalability, and so forth. It honors the enterprises and products that give outstanding performance in the enterprise-class IT market, providing a reference in the selection of products by enterprise ITs.

For more information, please see the following web page:

Continue reading
10161 Hits

A More Open Xen = A Stronger XenServer

[Originally posted on the Citrix blog by Tom McCafferty]

b2ap3_thumbnail_xen_project_logo.pngLast week, Citrix joined other industry leaders (Google, Cisco, Intel, Verizon) to create a collaborative open source project under the Linux Foundation to drive the future development of the Xen virtualization technology now named Xen Project. This is a significant development for the open source community at large, and was premiere news at the Linux Foundation Collaboration Conference earlier this month. The acceptance of into the Linux Foundation is also a testament to the dedication Citrix has put behind the Xen technology since acquiring XenSource in 2007.

Why make the move?
As we’ve moved into the Cloud Era, Citrix has found itself in the enviable position of having been the caretaker of the most successful hypervisor in the cloud. With Xen and XenServer now powering the majority of public cloud revenue in the world and growing interest as part of trend to low-power ARM processors, interest in Xen has been growingly rapidly.  This rapid project growth led the Xen project contributors to determine that the project and sub-projects would be better managed by a governing body like the Linux Foundation to provide the project a trusted, neutrally governed vehicle that can continue to drive its success.

The Linux Foundation is the natural home for this project as it is already home to a huge amount of industry collaboration and value creation, most notably with Linux but even more recently withOpenDaylight for the acceleration of SDN and the New York Stock Exchange led high performance Middleware Agnostic Messaging OpenMAMA project. Right out of the gate, big names have pledged their support to the new Xen Project. From mega clouds like Amazon Web Services, Terremark and Google, to leading advanced processor companies like Intel, AMD, and ARM/Calxeda, to virtualization platform providers such as Citrix and Oracle.

What does this mean for Citrix?
Xen provides the core hypervisor functionality for the XenServer virtualization platform which is a recognized industry leading solution used in datacenters around the globe. XenServer also provides the core platform technology for Citrix products like XenDesktop, NetScaler and CloudPlatform, which are powering thousands of clouds and thousands of datacenters today. With the creation of the new Xen Project and the guidance of the Linux Foundation, XenServer will now benefit from the development of other industry leaders along with informed input of the most demanding virtualization users on the planet.   Advancing the development of the underlying Xen hypervisor through better industry-wide collaboration strengthens our ability to ensure that XenServer will be a strong product and platform for future innovations as well.

This is an exciting time in the virtualization space where new opportunities and use cases are rewriting the rules for how a hypervisor should function and interoperate with datacenter infrastructure. Having an all-star cast of partners to work with on the newly dubbed Xen Project should provide all of the project supporters with powerful innovations that will shape the future of many of their respective open source and commercial efforts.  I for one am very excited about the impact this will have on Citrix XenServer and our users should be as well.

Stay on top of XenServer innovations with our Master Classes

Continue reading
18080 Hits

Transition from to XenProject

[Originally posted by Lars Kurth on the Xen blog].

This is just a quick note to you all, to outline the transition and timetable from the old main website to the new site. Other sites, such as,, and others will follow.

What will happen?

The site, will be moved to and all pages will clearly be marked as archived. We will redirect:

The intention is that web searches for xen and xen project will lead to the new website, while making sure that the old content is still available. Also, we want to ensure that linkx to will not be broken.

What has already happened?

We already created and set up the infrastructure for redirects. We also have a list of proposed redirects to We also removed all links to from

What is next?

We will be testing redirects on some of the less used pages in the coming days. Once we are confident, that all this works we will activate the redirects. The plan is to do this on June 8th.

How does this impact me?

Fundamentally, there should be no significant impact. If you check the Xen web pages regularly, you will be redirected to a corresponding new page or the archive.

If you have web real estate that links to, links will just be redirected to the new website. Although no links will be broken, you may want to …

  • Double check whether redirects go a location that matches your content
  • Ensure that links go to the new xenproject site (instead of the archived version)


If you are a vendor that is listed in the Xen Eco-system Pages, you are likely to see referrals from impacted. Thus, we would want to encourage you to create a new directory entries in the Xen Project Directory (which you can now do yourself, by creating a listing). We migrated static eco-system pages in the project and research categories (as these are more static). However we expect that other categories (Products, Consulting and Hosting) are owned (and changed as needed) by the respective vendors that provide a product or service.

Recent comment in this post
Can any one tell me free back up tools for XenServer 6.2.0 and also xendesktop..
Tuesday, 09 July 2013 12:07
Continue reading
8906 Hits
1 Comment

About XenServer

XenServer is the leading open source virtualization platform, powered by the Xen Project hypervisor and the XAPI toolstack. It is used in the world's largest clouds and enterprises.
Technical support for XenServer is available from Citrix.