Virtualization Blog

Discussions and observations on virtualization.

Introducing XenServer Dundee

It's spring time and after a particularly brutal winter here in Boston, I for one am happy to see the signs of spring. Grass and greenery, flowers budding, and warmer days all speak to good things coming. It's also time to unveil the next major XenServer project, code named Dundee. As with Creedence last year, we're going to be giving early access to a major new version of XenServer well in advance of its release. This project will have its share of functional improvements, and a few new features, but just like last year we're going to start with the platform and progress slowly.

CentOS 7 dom0

During the Creedence pre-release program, many commented on "Why CentOS 5.x? - CentOS 6 has been out for a while, and 7 is fresh". The answer to that question was pretty simple. We knew what userspace looked like with CentOS 5.x, and our users understood how to manage a CentOS 5.x system. CentOS 5 was being supported upstream until 2017, so there was no risk of us having something unsupported. Moving to CentOS 6.5 would've been a valid option if we didn't already plan on moving to CentOS 7, but we didn't want to change dom0 just to change it again in a years time. Plus if you recall, we took on quite a bit with Creedence in 2014.

So we're now a year later, and CentOS 7 makes perfect sense for dom0. Not only are there a few more upstream patches available, but Linux admins are now more comfortable with the changes in management paradigm. It's also those changes in paradigm which may present issues for you our users, and why this first alpha is all about validation. If you manage XenServer from a tool which uses the xapi SDK, then you shouldn't really experience too many problems. On the other hand, if you've favorite scripts, or tweaks you've made to configuration files, then you could be in for some extra work.

Now is also a perfect time to remind everyone that when you "upgrade" a XenServer, it's not an in place upgrade. We preserve the configuration files we know about, and then dom0 is reimaged. Any third party packages you have installed, custom scripts, and manual configuration changes have a good chance of being lost unless you've backed them up. In this case, with a move to CentOS 7, it's also possible that those items will need to be reworked to some degree.

Understanding the pre-release process

All pre-release downloads will be on our pre-release download page. We'll be providing new tagged builds every few weeks, and generally as we achieve internal milestones. With each build, we'll call out something which you as an interested participant in XenServer Dundee should be looking at. Issues encountered can be logged in the incident database at Since we've more than one version of XenServer covered in the incident database, please make certain you report Dundee issues under the "Dundee" version. Of course there is no guarantee we'll be able to resolve what you find, but we do want to know about it. With this first alpha, we’re interested in the “big issues” you may hit, i.e. areas which would block usage of features or functionality or cases where there is a major impact. These are really useful as the product develops and matures during the alpha stage. If you are developing something for XenServer, we invite you to ask your questions on the development mailing list, but do remember it's not a product support list.

Lastly, while we're in a pre-release period, its also likely you may eventually encounter functionality which may form part of a commercial edition. At this point we're not committing to what functionality will actually ship, when it might ship, or if it'll require a commercial license. I understand that might be concerning, but it shouldn't be. If something is destined for a commercial edition, you'll see it "commercialized" in a Citrix Tech Preview before we release. Historically we're many months away from when a Tech Preview might happen, so right now the most important thing is to focus on the changes we're interested in your feedback on today - everything to do with a CentOS 7 dom0.

Download Dundee alpha.1:

Recent Comments
Tobias Kreidl
Keep the XenServer evolution rolling! Great to see this initiative so soon after Creedence was released.
Tuesday, 28 April 2015 23:37
Tobias Kreidl
Is there a separate document available with just the release notes?
Wednesday, 29 April 2015 16:09
Martin Kralicek
Yes, the document/page containing roadmap or list of new features would be so good to have I am looking forward for next release.... Read More
Thursday, 07 May 2015 13:09
Continue reading
23198 Hits
10 Comments gets a refresh

Now that Creedence has shipped as XenServer 6.5, and we've even addressed some early issues with hotfixes (in record time no less), it was time to give a bit of an update as well. All of the content you've known to be on is still here, but this face lift is the first in a series of changes you'll see coming over the next few months.

Our Role

The role of will be shifting slightly from what we did in 2014 with an objective that by the end of 2015 it is the portal virtualization administrators use to find the information they need to be successful with XenServer. That's everything from development blogs, pre-release information, but also deeper technical content. Not everything will be hosted on, but we'll be looking for the most complete and accurate content available. Recognizing that commercial support is a critical requirement for production use of any technology, if we list a solution we'll also state clearly if its use is commercially supportable by Citrix or whether it could invalidate your support contract. In the end, this about successfully running a XenServer environment, so some practices presented might not be "officially sanctioned" and tested to the same level as commercially supported features, but are known by the community to work.

Community Content

The new will also have prominent community content. By its very nature, XenServer lives in a data center ecosystem populated by third party solutions. Some of those solutions are commercial in nature, and because commercial solutions should always retain "supported environment" status for a product, we've categorized them all under the "Citrix Ready" banner. Details on Citrix Ready requirements can be found on their FAQ page. Other solutions can be found within open source projects. We on the XenServer team are active in many, and we're consolidating information you'll need to be successful with various projects under the "Community" banner.

Commercial Content

We've always promoted commercial support on, and that's not changing. If anything, you'll see us bias a bit more towards promoting both support and some of the premium features within XenServer. After-all there is only one XenServer and the only difference between the installer you get from and from is the EULA. Once you apply a commercial license, or use XenServer as part of an entitlement within XenDesktop, you are bound by the same commercial EULA regardless of where the installation media originated.

Contributing Content

Public content contributions to have always been welcome, and with our new focus on technical information to run a successful XenServer installation, we're actively seeking more content. This could be in the form of article or blog submissions, but I'm willing to bet the most efficient way will be just letting us know about content you discover. If you find something, tweet it to me @XenServerArmy and we'll take a look at the content. If it is something we can use, we'll write a summary blog or article and link to it. Of course before that can happen we'll need to verify if the content could create an unsupported configuration and warn users upfront if it does.


What kind of content are we looking for? That's simple, anything you find useful to manage your XenServer installation. It doesn't matter how big or small that might be, or what tooling you have in place, if it helps you to be productive, we think that's valuable stuff for the community at large.     

Recent Comments
Tobias Kreidl
Tim, This is a great new direction and the planned diverse content is a welcome change. Many of the more technical articles have b... Read More
Thursday, 19 February 2015 19:15
Tim Mackey
I'm not certain what you mean by "font +/-". I used Chrome to scale the site and the fonts scaled as they should.
Wednesday, 25 February 2015 15:39
Tim Mackey
Thanks. I'm seeing something similar, and am curious if for you its just the homepage, or other pages?
Wednesday, 25 February 2015 15:38
Continue reading
16273 Hits

Creedence launches as XenServer 6.5

Today the entire XenServer team is very proud to announce that Creedence has officially been released as XenServer 6.5. It is available for download from, and is recommended for all new XenServer installs. We're so confident in what has been produced that I'm encouraging all XenServer 6.2 users to upgrade at their earliest convenience. So what have we actually accomplished?

The headline features

Every product release I've ever done, and there have been quite a large number over the years, has had some headline features; but Creedence is a bit different. Creedence wasn't about new features, and Creedence wasn't about chasing some perceived competitor. Creedence very much was about getting the details right for XenServer. It was about creating a very solid platform upon which anyone can comfortably, and successfully, build a virtualized data center regardless of workload. Creedence consisted of a lot of mundane improvements whose combination made for one seriously incredible outcome; Creedence restored the credibility of XenServer within the entire virtualization community. We even made up some t-shirts that the cool kids want ;)

So let's look at some of those mundane improvements, and see just how significant they really are.

  • 64 bit dom0 freed us from the limitations of dreaded Linux low memory, but also allows us to use modern drivers and work better with modern servers. From personal experience, when I took alpha.2 and installed it on some of my test Dell servers, it automatically detected my hardware RAID without my having to jump through any driver disk hoops. That was huge for me.
  • The move to a 3.10 kernel from meant that we were out of the business of having a completely custom kernel and corresponding patch queue. Upstream is goodness.
  • The move to the Xen Project hypervisor 4.4 meant that we're now consuming the most stable version of the core hypervisor available to us.
  • We've updated to an ovs 2.10 virtual switch giving us improved network stability when the virtual switch is under serious pressure. While we introduced the ovs way back in December of 2010, there remained cases where the legacy Linux bridge worked best. With Creedence, those situations should be very few and far between
  • A thread per vif model was introduced to better ensure network hogs didn't impact adjacent VM performance
  • Network datapath optimizations allow us to drive line rate for 10Gbps NICs, and we're doing pretty well with 40Gbps NICs.
  • Storage was improved through an update to tapdisk3, and the team did a fantastic job of engaging with the community to provide performance details. Overall we've seen very significant improvements in aggregate disk throughput, and when you're virtualizing it's the aggregate which matters more than the single VM case.

What this really means for you is that XenServer 6.5 has a ton more headroom than 6.2 ever did. If you happen to be on even older versions, you'll likely find that while 6.5 looks familiar, it's not quite like any other XenServer you've seen. As has been said multiple times in blog comments, and by multiple people, this is going to be the best release ever. In his blog, Steve Wilson has a few performance graphs to share for those doubters. 

The future

While today we've officially released Creedence, much more work remains. There is a backlog of items we really want to accomplish, and you've already provided a pretty long list of features for us to figure out how to make. The next project will be unveiled very soon, and you can count on having access to it early and being able to provide feedback just as the thousands of pre-release participants did for Creedence. Creedence is very much a success of the community as it is an engineering success.

Thank you to everyone involved. The hard work doesn't go unnoticed.     

Recent Comments
Finally! Now hope that it's as stable in production as it was during the testing phase... Fingers crossed here. BTW: There's a ty... Read More
Tuesday, 13 January 2015 19:32
Tim Mackey
@James, thanks for the kinds words @Fabian, I would expect things to be just as stable, and thanks for the catch.... Read More
Tuesday, 13 January 2015 19:39
Tobias Kreidl
Did DVSC ever get bundled in with the free or at least other versions?
Tuesday, 13 January 2015 19:42
Continue reading
51628 Hits

Creedence Release Candidate Available

Just in time for the holiday season - we're pleased to announce a another tech toy for the geeks of the world to play with. Now of course XenServer is serious business, but just like many kids toys, the arrival of Creedence is eagerly awaited by many. As I mentioned earlier this week, you'll have to wait a bit longer for the official release, but today you can download the release candidate and see exactly what the world of Creedence should look like. Andy also mentioned last week that we're closing out the alpha/beta program, and as part of that effort the nightly Creedence snapshot page has been removed. You can still access the final beta (beta.3) on the pre-release page, but all prior builds have been removed. The pre-release page is also where you can download the release candidate.

What's in the Release Candidate

Performance tuning

The release candidate contains a number of bug fixes, but also has had some performance tuning done on it. This performance tuning is a little bit different than what we normally talk about, so if you've been benchmarking Creedence, you'll want to double check with the release candidate. What we've done is take a look at the interaction of a variety of system components and put in some limits on how hard we'll let you push them. Our first objective is a rock solid system, and while this work doesn't result in any configuration limit changes (at least not yet - that comes later in our cycle), it could reduce some of the headroom you might have experienced with a prior build. It's also possible that you could experience better headroom due to an overall improvement in system stability, so doing a performance test or two isn't a bad idea.

Core bug fixes over beta.3

  • mulitpath.conf is now preserved as multipath.conf.bak on upgrade
  • The default cpufreq governor is now set to performance
  • Fixes for XSA-109 through XSA-114 inclusive
  • Increase the number of PIRQs to more than 256 to support large quantities of NICs per host

What we'd like you to do with this build

The core two things we'd like you to do with this build are:

  1. If you've reported any issue at, please validate that we did indeed get the issue addressed.
  2. If you can, run this release candidate through its paces. We think it's nice and solid, and hope you do too.

Lastly, I'd like to take this opportunity to wish everyone in our community a festive end to 2014 and hope that what ever celebrating you might do is enjoyable. 2014 was an exciting year for XenServer, and that's in large part to the contributions of everyone reading this blog and working with Creedence. Thank you.



Recent Comments
Tassos Papadopoulos
Boot from iSCSI is not supported on XS6.2. We had to do some hacks to do make it possible on Cisco UCS Blades. Are you going to su... Read More
Monday, 22 December 2014 07:21
Is "xe vm-export" performance improved in Creedence? From my test on RC, it looks similar, around 15MB/sec, even on 10GBE link: ... Read More
Monday, 22 December 2014 17:03
Tim Mackey
@Nathan, Since this is pre-release software intended for testing, no official upgrade path exists to the final release. Practica... Read More
Monday, 05 January 2015 19:53
Continue reading
15902 Hits

Status of Creedence

Over the past few weeks, and particularly as part of the Creedence World Tour, I've been getting questions about precisely when Creedence will be released. To the best of my ability, I've tried to take those questions head on, but the reality is we haven't been transparent about what happens when we release XenServer, and that's part of the problem. I'm going to try and address some of that in this post.

Now before I get into too much detail, it's important to note that XenServer is a packaged product which Citrix sells, and which is also available freely as open source. Citrix is a public company, so there is often a ton more detail I have, but which isn't appropriate for public disclosure. A perfect case in point is the release date. Since conceivably someone could change a PO based on this information, disclosing that can impact revenue and, well, I like my pay-cheque so I hope you understand when I'm quiet on some things.

So back to the question of what happens during finalization of a release, and how that can create a void. The first thing we do is take a look at all the defects coming in from all sources; with being one of many. We look at the nature of any open issues and determine what the potential for us to have a bad release from them. Next we create internal training to be delivered to the product support teams. These two tasks typically occur with either a final beta, or first release candidate. Concurrent to much of this work is finalization of documentation, and defining the performance envelope of the release. With each release, we have a "configuration limits" document, and the contents of that document represent both what Citrix is willing to deliver support on and what constitutes the limits of a stable XenServer configuration. For practical purposes, many of you have pushed Creedence in some way beyond what we might be comfortable defining as a "long term stable configuration", so its entirely possible the final performance could differ from what you've experienced so far.

Those are the technical bits behind the release, but this is also something which needs to be sold and that means we need to prepare for that as well. In the context of XenServer, selling means both why XenServer is great with XenDesktop, but also why it's great for anyone who is tired of paying more for their core virtualization requirements than really necessary. Given how many areas of data center operations XenServer touches, and the magnitude of the changes in Creedence, getting this right is critical. Then of course there is all the marketing collateral, and you get a sense of how much work is involved in getting XenServer out the door.

Of course, it can be argued that much of this "readiness" stuff could be handled in parallel, and for another project you'd be right. The reality is XenServer has had its share of releases which should've had a bit more bake time. I hope you agree with me that Creedence is better because we haven't rushed it, and that with Creedence we have a solid platform upon which to build. So with that in mind, I'll leave you with it hopefully being obvious that we intend to make a big splash with Creedence. Such a splash can't occur if we release during a typical IT lockdown period, and will need a bit larger stage than the one I'm currently on.


So stay tuned, my friends.  Good things are coming ;)

Recent Comments
Tim Mackey
Thanks @M.N..
Tuesday, 16 December 2014 14:42
tl;dr -- So what you're basically saying is that there is no release date yet.
Tuesday, 16 December 2014 12:03
Tim Mackey
@Christian It would be more precise to say that I know when it is intended to be released, but due to a variety of disclosure and... Read More
Tuesday, 16 December 2014 14:40
Continue reading
10436 Hits

Pushing XenServer limits with Creedence beta.2

Well folks, its that time once again; we've another XenServer build ripe for your inspection, and this one is a critical one for a number of reasons. Today we've released XenServer Creedence beta.2, and this is binary compatible with a Citrix Tech Preview refresh. The build number is 87850 and it presents itself to the outside world as 6.4.95. Over the past few announcements I've hinted at pushing the boundaries of XenServer and wanting the community at large to "have at it", but I've not put out too many details on the overall performance we're seeing internally. The most important attribute of this build is that internally, its going to form part of a series of long term stability tests. Yes folks, we're that confident in what we're seeing and I wanted to thank everyone who has participated in our pre-release activities by sharing a few performance tidbits:

  • Creedence can start and run 1000 PV VMs with only 8GB dom0 memory. That's up from the 650 we have in XenServer 6.2.
  • Booting 125 Windows 7 VMs on a single host takes only 350 seconds in a bootstorm scenario. That's down from 850 seconds in XenServer 6.2
  • Aggregate disk throughput has been measured to improve by as much as 100% when compared to XenServer 6.2
  • Aggregate intrahost network throughput has been measured to improve by as much as 200% when compared to XenServer 6.2
  • The number of virtual disks per host has been raised by a factor of four when compared to XenServer 6.2

When compared to beta.1, the team has been looking at a number of performance and scalability system aspects, with a primary focus on dom0 idle state behavior at scale. This is a very important aspect of system operation as overall system responsiveness is directly tied to the overhead of managing a large number of VMs. We did see two distinct areas for investigation, and are inviting the community to look into these and provide us with others. Those two areas are:

  • When using 40Gb NICs outbound (transmit) performance is below expectations. We have some internal fixes, but are encouraging anyone with such NICs to test and report their findings
  • When large numbers of hosts are pooled we're seeing VM start times appear to slow unexpectedly under large pool VM densities.


As always we're actively encouraging you to test the beta and provide your feedback (both positive and negative) in an incident report. You can download beta.2 from here:, and enter your feedback at     

Recent Comments
Tim Mackey
@bpbp If you're comparing the ovs in Creedence to previous Creedence builds, then they are identical. If you're comparing it to ... Read More
Sunday, 24 August 2014 23:37
Tim Mackey
@vati, If I understand correctly, you're looking for when the DVSC will appear? The DVSC is being handled via the Citrix Tech Pr... Read More
Monday, 25 August 2014 14:32
Tim Mackey
@bpbp, We're hoping that the upgrade to ovs 2.1.2 will address most of the issues seen with the older ovs. If you're in a positi... Read More
Monday, 25 August 2014 14:33
Continue reading
18676 Hits

XenServer Creedence Reaches Beta Stage

In May we announced to the world that the next version of XenServer would be code named Creedence, and officially opened a public alpha program around this new version of XenServer. That alpha program was more successful than I'd expected with well over a thousand participants. Drawing these participants to Creedence was a combination of enthusiasm for XenServer as a virtualization platform, and a desire to see us make significant improvements in that platform. What greeted them was a full platform refresh including a 64 bit dom0, modern Linux kernel and the most recent Xen Project hypervisor. Over the following weeks we invited this enthusiast community to test three additional builds, each with increasing capabilities and performance.

As the audience for the XenServer Creedence message increased, we heard from some that they wanted to wait until we exited alpha mode and entered a beta phase. I'm happy to report that we've done just that with the release on August 5th, 2014 of beta.1 for XenServer Creedence. This means we're largely feature complete, and are looking seriously at the overall performance and stability of the platform. It also means we want you to start stressing the platform with your real-world workloads. Everything is fair game at this point, and we want to know where the breaking points are. Over the coming weeks you'll see blog posts covering some of the key performance and scalability improvements that we've achieved internally, but that's internally. Your experiences will vary, and we want to know about them.

We want you to push XenServer and tell us where your expectations weren't met. Here's how to do just that:

  1. Download beta.1 and install it on your favorite hardware. If it doesn't install, we want to know. If it didn't detect the devices you have, we want to know. Oddly enough we also want to know if it does!!
  2. Create any number of VMs using your preferred operating systems, or alternatively use your favorite provisioning solution and provision some VMs. If something goes wrong here, let us know.
  3. Install into those VMs the applications you care about. Again if something goes wrong here, let us know.
  4. Exercise your applications, verify if the performance you see is what you'd expect. If it isn't let us know.
  5. While your applications are running, test core virtualization functions like live migration, storage migration, high availability, etc. Everything is fair game. If something doesn't behave as you expect, let us know.

How do you let us know of issues, you ask? Simply create a new account at and report an incident (or incidents). It's that simple. If you're looking to discuss at a deeper technical level why something might be behaving a certain way, or are seeking debugging help, then our developer list might be helpful. You can subscribe to it at:, but please understand that the developers are working hard to build XenServer and aren't product support folks.


We want Creedence to be the best XenServer release ever, and with your input it can be.     

Recent Comments
Tim Mackey
James, Thanks for the words of encouragement. beta.1 *should* be able to successfully upgrade 6.2 SP1 pools, and if it doesn't w... Read More
Wednesday, 06 August 2014 17:30
Bob Martens
Are we able to import raw Xen images?
Thursday, 07 August 2014 00:19
Tobias Kreidl
No Ceph support in Creedence, unfortunately. Keep lobbying for it! See: Read More
Wednesday, 13 August 2014 02:54
Continue reading
15392 Hits

XenServer Creedence Tech Preview and Creedence Alpha

This morning astute followers of XenServer activity noticed that Citrix had made available the previously announced Tech Preview for Creedence. The natural follow on question is how this relates to the alpha program we've been running on The easy answer is that the Citrix Tech Preview of XenServer Creedence is binary compatible with the alpha.4 pre-release binary you can get from From the perspective of the core platform (i.e. XenServer virtualization bits), the only difference is in the EULA.

So why run a Tech Preview if you have a successful alpha program already?

That's where the differences between a pure open source effort and a commercial effort begin. While the XenServer platform components are binary compatible, Citrix customers have expectations for features which couldn't be made open source, or implementations directly supporting Citrix commercial products. Perfect examples of these features and implementations can be seen on the Tech Preview download page, such as Microsoft System Center Integration, expanded vGPU support for XenDesktop, and the return of both the DVSC and WLB. While there is no guarantee any of those features or specific implementations will be present in the final Citrix release, or for that matter under what license, Citrix is seeking your input on them and a Tech Preview program is how that is accomplished.

I can't access the Tech Preview site; what's wrong?

If you can't login to the Tech Preview site, that likely means your account isn't associated with a Citrix commercial contract for XenServer. Since the alpha.4 pre-release is binary compatible, you can experience all the platform improvements yourself by downloading XenServer from the pre-release download page.

How do I provide feedback?

If you are able to participate in the Tech Preview program, you'll find the options for feedback listed on the Tech Preview page. Of course, even if you can participate in the Tech Preview program we're always accepting XenServer feedback through our public feedback options:

The XenServer incident database:

Development feedback (xs-devel):

What cool things are in alpha.4?

This is the best for last! We've already got in Creedence a 64 bit dom0, updated Linux 3.10 kernel, updated open virtual switch 2.1.2, improved boot storm handling, read caching for file based SRs; so what goodies are in here for the core platform people? Let's start with TRIM and UNMAP to better reclaim storage, then add in 32 bit to 64 bit VM migration to support upgrade scenarios and storage migration from XenServer 6.2 and prior, all with additional operating system validation with SLES 11 SP3 and Ubuntu 14.04 LTS.

What testing would you like us to do?

Don't let the alpha label fool you, alpha.4 of XenServer Creedence has been through quite a bit of testing, and is very much ready for you to try and stress it. The one thing we can say is that we're still working on the performance tuning so if you push things really hard, dom0 may run out of memory and you might need to follow CTX134951 to increase it (valid values are 8192, 16384 and 32768). This is particularly true if you're running more than 200 VMs per host, or need to attach more than 1200 virtual disks to VMs.     

Recent Comments
Tim Mackey
James, We now have the code in place to do a 32 bit to 64 bit migration, so upgrades are now on the table for testing, but I woul... Read More
Thursday, 10 July 2014 20:44
valentin radu
Hello, You can tell me when it will be released new stable version? Thx,
Tuesday, 15 July 2014 16:29
Hong Lae Cho
Hey Tim, Would you be able to provide a bit more clarification regarding "UNMAP" functionality in Creedence Alpha 4 & Tech Previ... Read More
Monday, 21 July 2014 05:10
Continue reading
26692 Hits

Whatever happened to XenServer's Windsor architecture?

b2ap3_thumbnail_Slide1.JPGAt the 2012 Xen Project Developer Summit in San Diego I talked about the evolution of XenServer's architecture, specifically our forward looking R&D work looking at a set of architectural changes known as "Windsor". The architecture includes a number of foundational overhauls, such as moving to a 64 bit domain-0 with a PVops kernel and upgrading to the upstream version of qemu (XenServer currently uses a forked Xen Project version and therefore doesn't benefit from new features and improvements made in the more active upstream project). Those of you following the development snapshots will have seen a number of these key component overhauls already.

The more notable changes in the new architecture include various forms of improved modularity within the system including "domain-0 disaggregation" as well as improved intra-component modularity and better internal APIs.

We wanted to do this for various reasons including:

  1. To improve single-host scalability (e.g. the number of VMs and the amount of aggregate I/O the system can sustain) by parallelizing the handling of I/O over a number of driver domains
  2. To enable better multi-host scalability in scale-out cloud environments, primarily by allowing each host to run more independently and therefore reduce the bottleneck effect of the pool master
  3. To create the capability to have additional levels of tenant isolation by having per-tenant driver domains etc.
  4. To allow for possible future third party service VMs (driver domains etc.)

So where are we at with this? In the single-host scalability area, something that Citrix customers care a lot about, we had a parallel effort to try to improve scale and performance in the short term by scaling up domain-0 (i.e. adding more vCPUs and memory) and tactically removing bottlenecks. We actually did better that we expected with this so it's reduced the urgency to build the "scale-out" disaggregated solution. Some of this works is described in Jonathan Davies' blog posts: How did we increase VM density in XenServer 6.2? and How did we increase VM density in XenServer 6.2? (part 2)

XenServer today does have some (officially unsupported) mechanisms to run driver domains. These have been used within Citrix in a promising evaluation of the use of storage drivers domains for a physical appliance running the Citrix CloudBridge product, performing significant amounts of caching related I/O to a very large number of local SSDs spread across a number of RAID controllers. This is an area where the scale-out parallelism of Windsor is well suited.

On the multi-host scalability side we've made some changes to both XenServer and Apache CloudStack (the foundation of the Citrix CloudPlatform cloud orchestration product) to reduce the load on the pool master and therefore make it possible to use the maximum resource pool size. For the longer term we're evaluating the overlap between XenServer's pool-based clustering and the various forms of host aggregation offered by orchestration stacks such as CloudStack and OpenStack. With the orchestration stacks' ability to manage a large number of hosts do we really need to indirect all XenServer commands through a pool master?

Disaggregation has taken place in the Xen Project XAPI toolstack used in XenServer. A prerequisite to moving the xapi daemon into a service VM was to split the higher level clustering and policy part of the daemon from the low level VM lifecycle management and hypervisor interface. From XenServer 6.1 the latter function was split into a separate daemon called xenopsd with the original xapi daemon performing the clustering and policy functions. In the network management part of the stack a similar split has been made to separate the network control function into xcp-networkd - this created immediate value by having a better defined internal API but is also a prerequisite for network driver domains. The current development version of the XAPI project has had a number of other modularity clean-ups including various services being split into separate daemons with better build and packaging separation.

b2ap3_thumbnail_demu.jpgWe're also using intra-component disaggregation for XenServer's virtual GPU (vGPU) support. A "discrete" emulator (DEMU) is used to provide the glue to allow the GPU vendor's control plane multiplexer driver in domain0 to service the control path parts of the vGPU access from the guest VM. This is done by, in effect, disaggregating qemu and having the DEMU take ownership of the I/O ports associated with the device it is emulating. This mechanism is now being added the the Xen Project to allow other virtual devices to be handled by discrete emulators, perhaps even in separate domains. Eventually we'd like to put the DEMUs and GPU driver into a driver domain to decouple the maintenance (particular required kernel version) of domain-0 and the GPU driver.

I view Windsor like a concept car, a way to try out new ideas and get feedback on their value and desirability. Like a concept car some of Windsor's ideas have made it into the shipping XenServer releases, some are coming, some are on the wishlist and some will never happen. Having a forward looking technology pipeline helps us to ensure that we keep evolving XenServer to meet users' needs both now and in the future.

Recent Comments
Tobias Kreidl
@James C: You can do that with Linux VMs -- and have been able to for years, it's just not supported by Citrix. We have run some s... Read More
Saturday, 03 May 2014 15:42
James Bulpin
We are looking at using the kernel mode block backend (blkback) for raw block devices, such as LVM (not LVHD) on SSDs and raw SAN ... Read More
Wednesday, 07 May 2014 16:47
Tobias Kreidl
We have done this for years on Linux boxes with directly-attached iSCSI connections, with significant performance gains. I wrote a... Read More
Saturday, 03 May 2014 16:10
Continue reading
22962 Hits

XenServer Status – January 2014

The release of true hardware GPU sharing and XenServer 6.2 SP1 was a strong finish to 2013 and based on the feedback from the Citrix Partner Summit a few weeks back, we really are a key differentiator for Citrix XenDesktop which fulfills one of the roles XenServer has in the ecosystem. Of course this also opens the question of how to get the sources for the cool new bits, and I’ll cover that in just a little bit. Looking beyond the commercial role for XenServer, we also saw significant growth in the core project with visitors, page views, downloads and mailing list activity all up at least 20% compared to December. From the perspective of engineering accomplishments, completed work in Q4 included a move to 4.3 for the Xen Project hypervisor, a move to CentOS 6.4 with Linux kernel to 3.10, and significant work towards a 64 bit dom0, upstream support for Windows PV drivers and blktap3 for storage. All told, this is a fantastic base from which to build upon in 2014.

Speaking of foundations, as an open source project we have an obligation to our community to provide clear access to the source used to produce XenServer. Unfortunately, it’s become apparent some confusion exists in the state of the project and source code locations. Fundamentally we had a miscommunication where many assumed the sources on and posted in GitHub represented XenServer 6.2 and that code changes which occurred in the GitHub repositories represented the XenServer 6.2 product direction. In reality, XenServer 6.2 represents a fork of XenServer which occurred prior to the creation of and the code which is part of represents trunk. So what does this mean for those of you looking for code, and for that matter test your solution with the correct binaries? To solve that I’ve created this handy little table:

XenServer 6.1 and prior Source is located on within the downloads section
XenServer 6.2 Source is located on within the downloads section, and on download page
XenServer 6.2 hotfixes Source is located within the zip file containing the hotfix
XenServer 6.2 SP1 Source is located within the zip file containing the service pack
XenServer trunk Source is located in the XenServer GitHub repository
XenServer nightly builds Source is located in the XenServer GitHub repository
XenCenter 6.1 and prior Source is not available
XenCenter 6.2 and later Source is located in the XenServer GitHub repository, and all XenCenter 6.2 versions are built from trunk
XenServer optional components Not all optional components are open source. For components which are open source, the source will be available with the component. Note that source code from third parties may require a license from the third party to obtain source (e.g. proprietary headers)


So what does this mean for specific feature work, and more importantly the next major version of XenServer? If the work being performed occurs within the XenServer 6.2 branch (for example as a hotfix), then that work will continue to be performed as it always has and source will be posted with that release. The only exception to that is work on XenCenter which is always occurring in trunk. Work for the next major release will occur in trunk as it currently has, but specific feature implementations in trunk shouldn't be considered "ready" until we actually release. In practice that means we may have some proof of concept stuff get committed, and we may decide that proof of concept work isn't compatible with newer work and refactor things before the release. I hope this clears things up a little, and there is now a better understanding of where a given feature can be found.     

Recent Comments
Tim, you mention the move to Xen 4.3, the move to CentOS 6.4 with Linux kernel to 3.10, and the significant work towards a 64-bit ... Read More
Sunday, 16 February 2014 06:52
Kristoffer Sheather
Can you provide a roadmap / projected schedule for the next releases of XenServer?
Monday, 03 March 2014 00:49
Continue reading
17734 Hits

XenServer Status – November 2013

The progress towards fulfilling the goal of making XenServer a proper open source project continues, but this month much of the work isn’t visible yet.  The big process improvements will hopefully be unveiled in late December or early January when we get our long needed wiki and defect trackers online.  The logical question of course is why it’s taking so long to get them out there.  After all we obviously do have the content, so why not just make it all public and be done?  Unfortunately, there is no magic wand to remove customer sensitive information, ensure that designs linked to closed source development on other Citrix products, or information provided to Citrix by partners under NDA isn’t accidentally made public.  Its painstaking work and we want to get it right.

In terms of partner announcements, we’ve been focusing on the NVIDIA vGPU work, as well as security efforts.

-          “Kaspersky trusted status” awarded to XenServer Windows Tools:

-          SAP 3D Enterprise on XenDesktop on XenServer powered by NVIDIA GRID:

-          The XenServer HCL has been expanded to include new servers from HP, Hitachi, Supermicro, Huawei, Lenovo and Fujitsu, storage devices from QNAP, Nexsan and Hitachi Data Systems, storage adapters from IBM and QLogic plus two CNAs from Emulex.

When I posted the project status last month, we had some significant gains, and this month is no different.  Compared to October:

-          Unique visitors were up 30% to 34,000

- page views were up 21% to over 110,000

-          Downloads of the XenServer installer were up by 7,000

-          We had over 110 commits to the XenServer repositories.

What’s most interesting about these stats isn’t the growth, which I do love, but that we’re getting to a point where the activity level is starting to feel right for a project of our maturity.  Don’t get me wrong, I still am looking for lots more growth, but I’m also looking for sustained activity.  That’s why I’m looking more at how XenServer interacts with its community, and what can be done to improve the relationship.  In my Open@Citrix blog, I asked the question “What kind of community do you want?”.  In my mind, everyone has a voice; it’s just up to you to engage with us.  I’d like to hear what you want from us, and that’s both the good and the bad.  If you have a community you’d like us to be involved with, I’d also like to hear about that too. 

Here is how I define the XenServer community:

The XenServer community is an independent group working to common purpose with a goal of leveraging each other to maximize the success of the community.  Members are proud to be associated with the community.


We all have a role to play in the future success of XenServer, and while I have the twitter handle of @XenServerArmy, I view my role as supporting you.  If there is something which is preventing you from adopting XenServer, or being as successful with XenServer as you intended, I want to know.  I want to remove as many barriers to adopting XenServer as I can, and I am your voice within the XenServer team at Citrix.  Please be vocal in your support, and vocal with what you need.

Recent Comments
srinivas j
XenServer status? Please post or update the status of XenServer roadmap
Sunday, 05 January 2014 03:31
srinivas j
currently hold XenServer licenses for 10+ hosts and are eagerly waiting for any updates to XenServer roadmap..
Sunday, 05 January 2014 03:32
Continue reading
12543 Hits

About XenServer

XenServer is the leading open source virtualization platform, powered by the Xen Project hypervisor and the XAPI toolstack. It is used in the world's largest clouds and enterprises.
Technical support for XenServer is available from Citrix.