Virtualization Blog

Discussions and observations on virtualization.

XenServer 6.5 SP1 Released

Wait, another XenServer release? Yes folks, there is no question we've been very busy improving upon XenServer over the past year, and the pace is quite fast. In case you missed it, we released XenServer 6.5 in January (formerly known as Creedence). Just a few weeks ago I announced and made available pre-release binaries for Dundee, and now we've just announced availability at Citrix Synergy of the first service pack for XenServer 6.5. Exciting times indeed.

What's in XenServer 6.5 SP1

I could bury the lead talk about hot fixes and roll-ups (more on that later), but the real value for SP1 is in the increased capabilities. Here are the lead items for this service pack:

  1. The Docker work we previewed in January at FOSDEM and later on xenserver.org is now available. If you've been using xscontainer in preview form, it should upgrade fine, but you should back up any VMs first. Completion of the Docker work also implies that CoreOS 633.1.0 is also an officially supported operating system with SP1. Containers deployed in Unbuntu 14.04 and RHEL, CentOS, and Oracle Enterprise Linux 7 and higher are supported.
  2. Adoption of LTS (long term support) guest support. XenServer guest support has historically required users of guest operating system to wait for XenServer to adopt official support for point releases in order to remain in a supported configuration. Starting with SP1, all supported operating systems can be upgraded within their major version and still retain "supported" status from Citrix support. For example, if a CentOS 6.6 VM is deployed, and the CentOS project subsequently releases CentOS 6.7, then upgrading that VM to CentOS 6.7 requires no changes to XenServer in order to remain a supported configuration.
  3. Intel GVT-d support for GPU pass through for Windows guests. This allows users of Xeon E3 Haswell processors to use the embedded GPU in those processors within a Windows guest using standard Intel graphics drivers.
  4. NVIDIA GPU pass though to Linux VMs allowing OpenGL and CUDA support for these operating systems.
  5. Installation of supplemental packs can now be performed through XenCenter Update. Note that since driver disks are only a special case of a supplemental pack, driver updates or installation of drivers not required for host installation can now also be performed using this mechanism
  6. Virtual machine density has been increased to 1000. What this means is that if you have a server which can reasonably be expected to run 1000 VMs of a given operating system, then using XenServer you can do so. No changes were made to the supported hardware configuration to accommodate this change.

Hotfix process

As with all XenServer service packs, XenServer 6.5 SP1 contains a rollup of all existing hot fixes for XenServer 6.5. This means that when provisioning a new host, your first post-installation step should be to apply SP1. It's also important to call out that when a service pack is released, hotfixes for the prior service pack level will no longer be created within six months. In this case, hotfixes for XenServer 6.5 will only be created through November 12th and following that point hotfixes will only be created for XenServer 6.5 SP1. In order for the development teams to streamline that transition, any defects raised for XenServer 6.5 in bugs.xenserver.org should be raised against 6.5 SP1 and not base 6.5.

Where to get XenServer 6.5 SP1

 

Downloading XenServer 6.5 SP1 is very easy, simply go to http://xenserver.org/download and download it!     

Recent Comments
Tobias Kreidl
Very nice! One quick question: It we want to work with both XS 6.5 SP1 and the technical pre-release Dundee version, is there or w... Read More
Tuesday, 12 May 2015 18:28
Tobias Kreidl
Found out these are 99% compatible (thanks, Andy!), so either should work fine. The Dundee XenCenter release shows a slightly newe... Read More
Wednesday, 13 May 2015 09:39
Stephen Turner
Hi, Tobias. Dundee XenCenter already includes all the changes in 6.5SP1, so you can just use that one to manage everything (everyt... Read More
Wednesday, 13 May 2015 08:23
Continue reading
35899 Hits
16 Comments

Introducing XenServer Dundee

It's spring time and after a particularly brutal winter here in Boston, I for one am happy to see the signs of spring. Grass and greenery, flowers budding, and warmer days all speak to good things coming. It's also time to unveil the next major XenServer project, code named Dundee. As with Creedence last year, we're going to be giving early access to a major new version of XenServer well in advance of its release. This project will have its share of functional improvements, and a few new features, but just like last year we're going to start with the platform and progress slowly.

CentOS 7 dom0

During the Creedence pre-release program, many commented on "Why CentOS 5.x? - CentOS 6 has been out for a while, and 7 is fresh". The answer to that question was pretty simple. We knew what userspace looked like with CentOS 5.x, and our users understood how to manage a CentOS 5.x system. CentOS 5 was being supported upstream until 2017, so there was no risk of us having something unsupported. Moving to CentOS 6.5 would've been a valid option if we didn't already plan on moving to CentOS 7, but we didn't want to change dom0 just to change it again in a years time. Plus if you recall, we took on quite a bit with Creedence in 2014.

So we're now a year later, and CentOS 7 makes perfect sense for dom0. Not only are there a few more upstream patches available, but Linux admins are now more comfortable with the changes in management paradigm. It's also those changes in paradigm which may present issues for you our users, and why this first alpha is all about validation. If you manage XenServer from a tool which uses the xapi SDK, then you shouldn't really experience too many problems. On the other hand, if you've favorite scripts, or tweaks you've made to configuration files, then you could be in for some extra work.

Now is also a perfect time to remind everyone that when you "upgrade" a XenServer, it's not an in place upgrade. We preserve the configuration files we know about, and then dom0 is reimaged. Any third party packages you have installed, custom scripts, and manual configuration changes have a good chance of being lost unless you've backed them up. In this case, with a move to CentOS 7, it's also possible that those items will need to be reworked to some degree.

Understanding the pre-release process

All pre-release downloads will be on our pre-release download page. We'll be providing new tagged builds every few weeks, and generally as we achieve internal milestones. With each build, we'll call out something which you as an interested participant in XenServer Dundee should be looking at. Issues encountered can be logged in the incident database at https://bugs.xenserver.org. Since we've more than one version of XenServer covered in the incident database, please make certain you report Dundee issues under the "Dundee" version. Of course there is no guarantee we'll be able to resolve what you find, but we do want to know about it. With this first alpha, we’re interested in the “big issues” you may hit, i.e. areas which would block usage of features or functionality or cases where there is a major impact. These are really useful as the product develops and matures during the alpha stage. If you are developing something for XenServer, we invite you to ask your questions on the development mailing list, but do remember it's not a product support list.

Lastly, while we're in a pre-release period, its also likely you may eventually encounter functionality which may form part of a commercial edition. At this point we're not committing to what functionality will actually ship, when it might ship, or if it'll require a commercial license. I understand that might be concerning, but it shouldn't be. If something is destined for a commercial edition, you'll see it "commercialized" in a Citrix Tech Preview before we release. Historically we're many months away from when a Tech Preview might happen, so right now the most important thing is to focus on the changes we're interested in your feedback on today - everything to do with a CentOS 7 dom0.

Download Dundee alpha.1: http://xenserver.org/preview

Recent Comments
Tobias Kreidl
Keep the XenServer evolution rolling! Great to see this initiative so soon after Creedence was released.
Tuesday, 28 April 2015 23:37
Tobias Kreidl
Is there a separate document available with just the release notes?
Wednesday, 29 April 2015 16:09
Martin Kralicek
Yes, the document/page containing roadmap or list of new features would be so good to have I am looking forward for next release.... Read More
Thursday, 07 May 2015 13:09
Continue reading
22421 Hits
10 Comments

Participe do XenServer Day Fortaleza 2015

Participe do XenServer Day Fortaleza 2015

Fortaleza | Ceará | Brasil | 27/02/2015 | 14:00h
UFC - Universidade Federal do Ceará Campus do Pici | Bloco 902 |
Auditório Reitor Ícaro de Souza Moreira (Auditório do Centro de Ciências)

O XenServer Day foi um evento criado para usuários corporativos, desenvolvedores, fornecedores de serviços e entusiastas pelo Citrix™ XenServer™. O evento será realizado na Cidade de Fortaleza/CE, na Universidade Federal do Ceará, Auditório Reitor Ícaro de Souza Moreira (Auditório do Centro de Ciências), no dia 27/02/2015. Nesta edição do XenServer Day, vai ser realizada em conjunto com o XenServer Creedence World Tour, iniciativa da comunidade Open Source XenServer.org para lançamento do XenServer 6.5 (Codename Creedence).

Segue abaixo agenda de palestras:

14:00h - Abertura do Evento
14:20h - O que há de novo no Citrix XenServer 6.5 - Lorscheider Santiago (Quales Tecnologia)
15:30h - Palestra Unitrends - André Favoretto (Globix)
16:10h - Gerenciando infraestruturas virtuais em nuvem, servidor e desktop com Citrix XenServer 6.5  - Lorscheider Santiago (Quales Tecnologia)
17:00h - Infraestrutura de TI com Segurança: O que isso representa para o seu negócio - Vinícius Minneto (Ascenty)
17:30h - Encerramento

Os primeiros participantes que chegarem ao evento vão ganhar a camisa oficial da XenServer Creedence World Tour (Estoque limitado)

Para mais informações sobre o evento e fazer a sua inscrição, clique aqui

Está nas Redes Sociais? Compartilhe a hashtag: #XenServerDayFortaleza

Lorscheider Santiago - @lsantiagos

Recent comment in this post
Hafiz
Hi, I am attempting to create a virtual windows 2012 r2 server in Citrix XenServer 7.6 but keep getting an error during the initi... Read More
Saturday, 28 November 2015 00:55
Continue reading
8387 Hits
1 Comment

XenServer Support Options

Now that Creedence has been released as XenServer 6.5, I'd like to take this opportunity to highlight where to obtain what level of support for your installation.

Commercial Support

Commercial support is available from Citrix and many of its partners. A commercial support contract is appropriate if you're running XenServer in a production environment, particularly if downtime is a critical component of your SLA. It's important to note that commercial support is only available if the deployment follows the Citrix deployment guidelines, uses third party components from the Citrix Ready Marketplace, and is operated in accordance with the terms of the commercial EULA. Of course, since your deployment might not precisely follow these guidelines, commercial support may not be able to resolve all issues and that's where community support comes in.

Community Support

Community support is available from the Citrix support forums. The people on the forum are both Citrix support engineers and also your fellow system administrators. They are generally quite knowledgeable and enthusiastic to help someone be successful with XenServer. It's important to note that while the product and engineering teams may monitor the support forums from time to time, engineering level support should not be expected on the community forums.

Developer Support

Developer level support is available from the xs-devel list. This is your traditional development mailing list and really isn't appropriate for general support questions. Many of the key engineers are part of this list, and do engage on topics related to performance, feature development and code level issues. It's important to remember that the XenServer software is actually built from many upstream components, so the best source of information might be an upstream developer list and not xs-devel.

Self-support tool

Citrix maintains an self-support tool called Citrix Insight Services, formerly known as Tools-as-a-Service (TaaS). Insight Services takes a XenServer status report, and analyzes it to determine if there are any operational issues present in the deployment. A best practice is to upload a report after installing a XenServer host to determine if any issues are present which can result in latent performance or stability problems. CIS is used extensively by the Citrix support teams, but doesn't require a commercial support contract for end users.

Submitting Defects

If you believe you have encountered a defect or limitation in the XenServer software, simply using one of these support options isn't sufficient for the incident to be added to the defect queue for evaluation. Commercial support users will need to have their case triaged and potentially escalated, with the result potentially being a hotfix. All other users will need to submit an incident report via bugs.xenserver.org. Please be as detailed as possible with any defect reports such that they can be reproduced, and it doesn't hurt to include the URL of any forum discussion or the TaaS ID in your report. Also, please be aware that while the issue may be urgent for you any potential fix may take some time to be created. If your issue is urgent, you are strongly encouraged to follow the commercial support route as Citrix escalation engineers have the ability to prioritize customer issues.

Additionally, its important to point out that submitting a defect or incident report doesn't guarantee it'll be fixed. Some things simply work the way they do for very important reasons, other things may behave the way they do due to the way components interact. XenServer is tuned to provide a highly scalable virtualization platform, and if an incident would require destabilizing that platform, it's unlikely to be changed.

Recent comment in this post
JK Benedict
Tim - thank you very much for this post and as always, we greatly appreciate your work here @ xenserver.org. #XenServer = #Citr... Read More
Friday, 16 January 2015 04:37
Continue reading
15738 Hits
1 Comment

Creedence launches as XenServer 6.5

Today the entire XenServer team is very proud to announce that Creedence has officially been released as XenServer 6.5. It is available for download from xenserver.org, and is recommended for all new XenServer installs. We're so confident in what has been produced that I'm encouraging all XenServer 6.2 users to upgrade at their earliest convenience. So what have we actually accomplished?

The headline features

Every product release I've ever done, and there have been quite a large number over the years, has had some headline features; but Creedence is a bit different. Creedence wasn't about new features, and Creedence wasn't about chasing some perceived competitor. Creedence very much was about getting the details right for XenServer. It was about creating a very solid platform upon which anyone can comfortably, and successfully, build a virtualized data center regardless of workload. Creedence consisted of a lot of mundane improvements whose combination made for one seriously incredible outcome; Creedence restored the credibility of XenServer within the entire virtualization community. We even made up some t-shirts that the cool kids want ;)

So let's look at some of those mundane improvements, and see just how significant they really are.

  • 64 bit dom0 freed us from the limitations of dreaded Linux low memory, but also allows us to use modern drivers and work better with modern servers. From personal experience, when I took alpha.2 and installed it on some of my test Dell servers, it automatically detected my hardware RAID without my having to jump through any driver disk hoops. That was huge for me.
  • The move to a 3.10 kernel from kernel.org meant that we were out of the business of having a completely custom kernel and corresponding patch queue. Upstream is goodness.
  • The move to the Xen Project hypervisor 4.4 meant that we're now consuming the most stable version of the core hypervisor available to us.
  • We've updated to an ovs 2.10 virtual switch giving us improved network stability when the virtual switch is under serious pressure. While we introduced the ovs way back in December of 2010, there remained cases where the legacy Linux bridge worked best. With Creedence, those situations should be very few and far between
  • A thread per vif model was introduced to better ensure network hogs didn't impact adjacent VM performance
  • Network datapath optimizations allow us to drive line rate for 10Gbps NICs, and we're doing pretty well with 40Gbps NICs.
  • Storage was improved through an update to tapdisk3, and the team did a fantastic job of engaging with the community to provide performance details. Overall we've seen very significant improvements in aggregate disk throughput, and when you're virtualizing it's the aggregate which matters more than the single VM case.

What this really means for you is that XenServer 6.5 has a ton more headroom than 6.2 ever did. If you happen to be on even older versions, you'll likely find that while 6.5 looks familiar, it's not quite like any other XenServer you've seen. As has been said multiple times in blog comments, and by multiple people, this is going to be the best release ever. In his blog, Steve Wilson has a few performance graphs to share for those doubters. 

The future

While today we've officially released Creedence, much more work remains. There is a backlog of items we really want to accomplish, and you've already provided a pretty long list of features for us to figure out how to make. The next project will be unveiled very soon, and you can count on having access to it early and being able to provide feedback just as the thousands of pre-release participants did for Creedence. Creedence is very much a success of the community as it is an engineering success.

Thank you to everyone involved. The hard work doesn't go unnoticed.     

Recent Comments
Fabian
Finally! Now hope that it's as stable in production as it was during the testing phase... Fingers crossed here. BTW: There's a ty... Read More
Tuesday, 13 January 2015 19:32
Tim Mackey
@James, thanks for the kinds words @Fabian, I would expect things to be just as stable, and thanks for the catch.... Read More
Tuesday, 13 January 2015 19:39
Tobias Kreidl
Did DVSC ever get bundled in with the free or at least other versions?
Tuesday, 13 January 2015 19:42
Continue reading
49878 Hits
51 Comments

Understanding why certain Creedence builds don't work with certain features

Over the year end break, there were a couple of posts to the list which asked a very important question: "Does the DVSC work with the Release Candidate?" The answer was a resounding "maybe", and this post is intended to help clarify some of the distinction between what you get from xenserver.org, what you get from citrix.com, and how everything is related.

At this point most of us are already familiar with XenServer virtualization being "open source", and that with XenServer 6.2 there was no functional difference between the binary you could download from citrix.com and that from xenserver.org. Logically, when we started the Creedence pre-release program, many assumed that the same download would exist in both locations, and that everything which might be part of a "XenServer" would also always be open source. That would be really cool for many people, and potentially problematic for others.

The astute follower of XenServer technology might also have noticed that several things commonly associated with the XenServer product never had their source released. StorageLink is a perfect example of this. Others will have noticed that the XenServer Tech Preview run on citrix.com included a number of items which weren't present in any of the xenserver.org pre-release builds, and for which the sources aren't listed on xenserver.org. There is of course an easy explanation for this, but it goes to the heart of what we're trying to do with xenserver.org.

xenserver.org is first and foremost about the XenServer platform. Everyone associated with xenserver.org, and by extension the entire team, would love for the data centers of the world to standardize on this platform. The core platform deliverable is called main.iso, and that's the thing from which you install a XenServer host. The source for main.iso is readily available, and other than EULA differences, the XenServer host will look and behave identically regardless of whether main.iso came from xenserver.org or citrix.com. The beauty of this model is that when you grow your XenServer based data center to the point where commercial support makes sense, the software platform you'd want supported is the same.

All of which gets me back to the DVSC (and other similar components). DVSC, StorageLink and certain other "features" include source code which Citrix has access to under license. Citrix provides early access to these feature components to those with a commercial relationship. Because there is no concept of a commercial relationship with xenserver.org, we can't provide early access to anything which isn't part of the core platform. Now of course we do very much want everyone to obtain the same XenServer software from both locations, so when an official release occurs, we mirror it for your convenience.

I hope this longish explanation helps clarify why when questions arise about "features" not present in main.iso that the response isn't as detailed as some might like. It should also help explain why components obtained from prior "Tech Preview" releases might not work with newer platform builds obtained as part of a public pre-release program.

Recent Comments
Tim Mackey
@xiao, It went live this morning, and you can download it from xenserver.org
Tuesday, 13 January 2015 19:40
Tim Mackey
@Nick I think a better way of thinking about this is that the hypervisor is free, and the platform features and functions are fre... Read More
Tuesday, 27 January 2015 23:43
Continue reading
14658 Hits
4 Comments

Creedence Release Candidate Available

Just in time for the holiday season - we're pleased to announce a another tech toy for the geeks of the world to play with. Now of course XenServer is serious business, but just like many kids toys, the arrival of Creedence is eagerly awaited by many. As I mentioned earlier this week, you'll have to wait a bit longer for the official release, but today you can download the release candidate and see exactly what the world of Creedence should look like. Andy also mentioned last week that we're closing out the alpha/beta program, and as part of that effort the nightly Creedence snapshot page has been removed. You can still access the final beta (beta.3) on the pre-release page, but all prior builds have been removed. The pre-release page is also where you can download the release candidate.

What's in the Release Candidate

Performance tuning

The release candidate contains a number of bug fixes, but also has had some performance tuning done on it. This performance tuning is a little bit different than what we normally talk about, so if you've been benchmarking Creedence, you'll want to double check with the release candidate. What we've done is take a look at the interaction of a variety of system components and put in some limits on how hard we'll let you push them. Our first objective is a rock solid system, and while this work doesn't result in any configuration limit changes (at least not yet - that comes later in our cycle), it could reduce some of the headroom you might have experienced with a prior build. It's also possible that you could experience better headroom due to an overall improvement in system stability, so doing a performance test or two isn't a bad idea.

Core bug fixes over beta.3

  • mulitpath.conf is now preserved as multipath.conf.bak on upgrade
  • The default cpufreq governor is now set to performance
  • Fixes for XSA-109 through XSA-114 inclusive
  • Increase the number of PIRQs to more than 256 to support large quantities of NICs per host

What we'd like you to do with this build

The core two things we'd like you to do with this build are:

  1. If you've reported any issue at https://bugs.xenserver.org, please validate that we did indeed get the issue addressed.
  2. If you can, run this release candidate through its paces. We think it's nice and solid, and hope you do too.

Lastly, I'd like to take this opportunity to wish everyone in our community a festive end to 2014 and hope that what ever celebrating you might do is enjoyable. 2014 was an exciting year for XenServer, and that's in large part to the contributions of everyone reading this blog and working with Creedence. Thank you.

 

-tim     

Recent Comments
Tassos Papadopoulos
Boot from iSCSI is not supported on XS6.2. We had to do some hacks to do make it possible on Cisco UCS Blades. Are you going to su... Read More
Monday, 22 December 2014 07:21
Itaru OGAWA
Is "xe vm-export" performance improved in Creedence? From my test on RC, it looks similar, around 15MB/sec, even on 10GBE link: ... Read More
Monday, 22 December 2014 17:03
Tim Mackey
@Nathan, Since this is pre-release software intended for testing, no official upgrade path exists to the final release. Practica... Read More
Monday, 05 January 2015 19:53
Continue reading
15475 Hits
15 Comments

Status of Creedence

Over the past few weeks, and particularly as part of the Creedence World Tour, I've been getting questions about precisely when Creedence will be released. To the best of my ability, I've tried to take those questions head on, but the reality is we haven't been transparent about what happens when we release XenServer, and that's part of the problem. I'm going to try and address some of that in this post.

Now before I get into too much detail, it's important to note that XenServer is a packaged product which Citrix sells, and which is also available freely as open source. Citrix is a public company, so there is often a ton more detail I have, but which isn't appropriate for public disclosure. A perfect case in point is the release date. Since conceivably someone could change a PO based on this information, disclosing that can impact revenue and, well, I like my pay-cheque so I hope you understand when I'm quiet on some things.

So back to the question of what happens during finalization of a release, and how that can create a void. The first thing we do is take a look at all the defects coming in from all sources; with bugs.xenserver.org being one of many. We look at the nature of any open issues and determine what the potential for us to have a bad release from them. Next we create internal training to be delivered to the product support teams. These two tasks typically occur with either a final beta, or first release candidate. Concurrent to much of this work is finalization of documentation, and defining the performance envelope of the release. With each release, we have a "configuration limits" document, and the contents of that document represent both what Citrix is willing to deliver support on and what constitutes the limits of a stable XenServer configuration. For practical purposes, many of you have pushed Creedence in some way beyond what we might be comfortable defining as a "long term stable configuration", so its entirely possible the final performance could differ from what you've experienced so far.

Those are the technical bits behind the release, but this is also something which needs to be sold and that means we need to prepare for that as well. In the context of XenServer, selling means both why XenServer is great with XenDesktop, but also why it's great for anyone who is tired of paying more for their core virtualization requirements than really necessary. Given how many areas of data center operations XenServer touches, and the magnitude of the changes in Creedence, getting this right is critical. Then of course there is all the marketing collateral, and you get a sense of how much work is involved in getting XenServer out the door.

Of course, it can be argued that much of this "readiness" stuff could be handled in parallel, and for another project you'd be right. The reality is XenServer has had its share of releases which should've had a bit more bake time. I hope you agree with me that Creedence is better because we haven't rushed it, and that with Creedence we have a solid platform upon which to build. So with that in mind, I'll leave you with it hopefully being obvious that we intend to make a big splash with Creedence. Such a splash can't occur if we release during a typical IT lockdown period, and will need a bit larger stage than the one I'm currently on.

 

So stay tuned, my friends.  Good things are coming ;)

Recent Comments
Tim Mackey
Thanks @M.N..
Tuesday, 16 December 2014 14:42
Christian
tl;dr -- So what you're basically saying is that there is no release date yet.
Tuesday, 16 December 2014 12:03
Tim Mackey
@Christian It would be more precise to say that I know when it is intended to be released, but due to a variety of disclosure and... Read More
Tuesday, 16 December 2014 14:40
Continue reading
9963 Hits
6 Comments

XenServer Pre-Release Programme

A very big thank you for everyone who participated in the Creedence Alpha/Beta programme! 
The programme was very successful and raised a total of 177 issues, of which 138 were resolved during the Alpha/Beta period.  We are reviewing how the pre-release process can be improved and streamlined going forward. 

The Creedence Alpha/Beta programme has now come to an end with the focus of nightly snapshots moving on to the next version of XenServer.   

The Creedence Alpha/Beta source code remains available and can be accessed here: 
http://xenserver.org/component/content/article/24-product/creedence/143-xs-2014-development-snapshots.html

Creedence Alpha/Beta bugs may still be reported on https://bugs.xenserver.org

Work is already progressing on the next version of XenServer and the nightly snapshots are available here:
http://xenserver.org/component/content/article/2-uncategorised/115-development-snapshots.html

As this work is new and still expected to be unstable, please do not raise any Creedence Alpha/Beta bugs against it.

Recent Comments
Andrew Halley
Hi there, we are working towards posting an updated build containing all the bug fixes received to date, and which is fully integr... Read More
Sunday, 14 December 2014 12:33
Tobias Kreidl
Andrew, Thanks go to you and the whole Citrix team for making this a really great overall experience. Each XenServer release seems... Read More
Wednesday, 10 December 2014 19:45
Andrew Halley
Appreciate it Tobias - and our thanks to all the excellent contributions received from our community.
Sunday, 14 December 2014 12:33
Continue reading
15471 Hits
25 Comments

Increasing Ubuntu's Resolution

Increasing Ubuntu's Resolution

Maximizing Desktop Real-estate with Ubuntu

With the addition of Ubuntu (and the likes) to Creedence, you may have noticed that the default resolution is 1024x768.  I certainly noticed it and with much work on 6.2 and Creedence Beta, I have a quick solution to maximizing the screen resolution for you.

The thing to consider is that a virtual frame buffer is what is essentially being used.  You can re-invent X configs all day, but the shortest path is to - first - ensure that that the following files are installed on your Ubuntu guest VM:

sudo apt-get install xvfb xfonts-100dpi xfonts-75dpi xfstt

Once that is all done installing, the next step is to edit Grub -- specifically /etc/default/grub:

sudo vi /etc/default/grub

Considering your monitor's maximum resolution (or not if you want to remote into Ubuntu using XRDP), look for the variable GRUB_GFXMODE.  This is where you can specify your desired BOOT resolutions that we will instruct the guest VM to SUSTAIN into user-space:

GRUB_GFXMODE=1280x960,1280x800,1280x720,1152x768,1152x700,1024x768,800x600

Next, adjust the variable GRUB_PAYLOAD_LINUX to equal keep, or:

GRUB_PAYLOAD_LINUX=keep

Save the changes and be certain to execute the following:

sudo update-grub
sudo reboot

Now, you will notice that even during the boot phase that the resolution is large and this will carry into user space: Lightdm, Xfce, and the likes.

Finally, I would highly suggest installing XRDP for your Guest VM.  It allows you to access that Ubuntu/Xbunutu/etc desktop remotely.  Specific details regarding this can be found through Ubuntu's forum:

http://askubuntu.com/questions/449785/ubuntu-14-04-xrdp-grey


Enjoy!

--jkbs | @xenfomation

 

 

Recent Comments
JK Benedict
Thanks, YLK - I am so glad to hear this helped someone else! Now... install XRDP and leverage the power to Remote Desktop (secure... Read More
Thursday, 25 December 2014 04:46
gfpl
thanks guy is very good help me !!!
Friday, 06 March 2015 10:52
Fredrik Wendt
Would be really nice to see all steps needed (CLI on dom0) to go from http://se.archive.ubuntu.com/ubuntu/dists/vivid/main/install... Read More
Monday, 14 September 2015 21:48
Continue reading
25912 Hits
6 Comments

Creedence: Debian 7.x and PVHVM Testing

Introduction

On my own time and on my own testing equipment, I have been able to run many Guests VMs in PVHVM containers - before Creedence after its release to the public back in June.  Last week's broadcast of Creedence Beta 3's release, I was naturally excited to see Tim's spotlight on PVHVM and the following article's intent is to show - in a test environment only - how I was able to run Debian 7.x (64-bit) in the same fashion.

For more information regarding PV + HVM as to establish a PVHVM container, Tim linked a great article in his Creedence Beta 3 post last Monday that I highly recommend you read as the finer details are out of scope for this article's intent and purpose.

Why is this important to me?  Quite simply we can go from this....

... to this ...

So now, let's make a PVHVM container for a Debian 7.x (64-Bit) Guest VM within XenCenter!

Requirements

1.  Creedence Beta 3 and XenCenter

2.  The full installation ISO for Debian 7.x (from https://www.debian.org/CD/http-ftp/#stable )

3.  Any changes mentioned below should not be applied to any of the stock Debian templates

4.  This should not be performed on your production environment

Creating A Default Template

With XenCenter open, ensure that from the View options one has "XenServer Templates" selected:

We should now see the default templates that XenServer installs:

1.  Right-click on the "Debian Wheezy 7 (64-bit)" template and save it as "Debian 7":

 

3.  This will produce a "custom template" - highlight it and copy the UUID of the custom template:

4.  The remainder of this configuration will take place from the command-line.

5.  To make the changes to the custom template easier, export the UUID of the custom template we created to avoid copy/paste errors:

export myTemp="af84ad43-8caf-4473-9c4d-8835af818335"
echo $myTemp
af84ad43-8caf-4473-9c4d-8835af818335

6.  With the $myTemp variable created, let us first convert this custom template to a default template by executing:

xe template-param-set uuid=$myTemp other-config:default_template=true

xe template-param-remove uuid=$myTemp param-name=other-config param-key=base_template_name

7.  Now configure the template's "platform" variable to leverage VGA graphics:

xe template-param-set uuid=$myTemp platform:viridian=false platform:device_id=0001 platform:vga=std platform:videoram=16

8.  Due to how some distros work with X, clear the PV-args and set a "vga=792" flag:

xe template-param-set uuid=$myTemp PV-args="vga=792"

9.  Disable the PV-bootloader:

xe template-param-set uuid=$myTemp PV-bootloader=""

10.  Specify that the template uses an HVM-style bootloader (DVD/CD first, then hard drive, and then network):

xe template-param-set uuid=$myTemp HVM-boot-policy="BIOS order"
xe template-param-set uuid=$myTemp HVM-boot-params:order="dcn"

 

Now, before creating a Debian 7.x Guest VM, one should now see in XenCenter that "Debian 7" is listed as a "default template":

 

Lastly, for the VGA flag and what it means to most distros, the following is a table explaining the VGA flag and bit settings to achieve XxY resoluton @ a color depth:

VGA Resolution and Color Depth reference Chart:

Depth 800×600 1024×768 1152×864 1280×1024 1600×1200
8 bit vga=771 vga=773 vga=353 vga=775 vga=796
16 bit vga=788 vga=791 vga=355 vga=794 vga=798
24 bit vga=789 vga=792   vga=795 vga=799

Create A New Debian Guest

From now, one should be able to create a new Guest VM using the template we have just created and should be able to walk through the entire install:

Post installation, tools can be installed as well!

Enjoy and happy testing!

 

jkbs | @xenfomation

Recent Comments
JK Benedict
Hey, Tobi - Thanks for the feedback! With regards to the graphical install, are you referring to how to do this with XenServer 6... Read More
Friday, 10 October 2014 19:40
JK Benedict
Alrighty -- Been busy, but the following BASH script should make a copy of your Debain 7 template and make a generic, HVM templat... Read More
Wednesday, 22 October 2014 03:10
JK Benedict
You should quite able to copy-n-paste the code above -- that will remove the emoticons from the colon + some other character.... Read More
Wednesday, 22 October 2014 03:21
Continue reading
20987 Hits
18 Comments

Creedence Final Beta Available

As we move steadily towards a release of XenServer Creedence, I'm pleased to announce that we're ending the beta phase of development with the release of Creedence beta.3. Beta.3 sees us as functionally complete, and with the majority of known performance issues resolved. The performance issues resolved range from a dom0 memory leak when VIFs are disabled, through to resolution of a workaround with Mellanox 40Gbps NICs, and some are resolved with both an updated driver bundle and a bump of the ovs version from 2.1.2 to 2.1.3. Functionally, beta.3 differs from beta.2 in having PVHVM support for Ubuntu 14.04, RedHat Enterprise Linux 7 and CentOS 7. Since these are new operating systems for us, the team is really interested in learning what you see for performance and stability for them.

As with all previous pre-release builds, we'd like the community to help ensure Creedence is a rock solid release. This time we're a bit less interested in Creedence itself, and more about the operating and support environment. One of the less known "features" of Citrix support is the free "Insight Services" or "TaaS". TaaS was originally designed to be "Tools-as-a-Service", and deliver on demand insight into the operation of Citrix technologies. With XenServer, Citrix Insight Services consumes a server status report from your XenServer host or pool, and then provides detailed guidance on how to potentially avoid an issue (say due to outdated BIOS or firmware), or resolve an issue you might be having (say by applying a hotfix). Honestly, it's not a bad practice in general to upload a server status report post XenServer install to ensure there aren't any items which could be latent in the deployment; rather like a health check.

How does this relate to Creedence? Well, Insight Services uses a series of plugins to ensure the data is processed properly. The support team has recently updated TaaS to support Creedence, and I'd like to ensure two things. First I'd like to ensure the processing logic is capturing everything it should, and secondly I'd like to ensure that those of you who have been successfully running Creedence don't have any hidden errors. Since this is a free service offered by Citrix, I'd also like the open source XenServer install base to know about it as a way to ensure XenServer hosts are deployed in a manner which will allow for Citrix to support you if the need arises.

Here's how you can help.

  1. Install either beta.2 or beta.3 (beta.3 preferred) from the pre-release downloads: http://xenserver.org/overview-xenserver-open-source-virtualization/prerelease.html
  2. From either XenCenter or the CLI take a server status report.
    • XenCenter: Server status reports can be run from "Tools->Server Status Report ..."
    • CLI: xen-bugtool --yestoall --output zip
  3. Log into TaaS (create a free account if required): https://taas.citrix.com/AutoSupport/
  4. Upload your server status report and see if anything interesting is found. If anything unexpected is found, we'd like to know about it.  The best way to let us know would be to submit an incident to https://bugs.xenserver.org which contains the TaaS information.

Thanks again to everyone who has contributed to the success we're seeing with Creedence.

Recent Comments
JK Benedict
Great post, Tim! I will definitely be leveraging this tomorrow after I check in my previous posts for PVHVM OSes I have used from... Read More
Thursday, 25 September 2014 00:42
Sebastian
hmm.. ? should yum install work ? Loaded plugins: fastestmirror Determining fastest mirrors Could not retrieve mirrorlist http://... Read More
Friday, 26 September 2014 15:14
Tim Mackey
@Sebastian, XenServer is a packaged and tuned system, so we intentionally disable yum install. The biggest reason we do that is ... Read More
Friday, 26 September 2014 15:47
Continue reading
18035 Hits
20 Comments

Post-Creedence details to be presented at Xen Project User Summit

Last month I posted seeking feedback from you our community on what the post-Creedence world should look like. The response was impressive, and we've started incorporating what you want in a virtualization platform into our plans for the next release of XenServer; occurring after Creedence. While it's a bit early to divulge those details, I plan as part of my session on Creedence at the Xen Project User Summit to give you a roadmap for what to expect, what the code name will be for the project, and how you can help move the project forward. There will also be a few other surprises at the event for XenServer attendees, so if are able to be in New York City on September 15th please do try and join us. Not only will you be able to see what the new XenServer has to offer, but you'll see what the core hypervisor community is up to and potentially push them to help deliver features you feel valuable.     

Recent Comments
Russell Pavlicek
And don't forget that the leaders of the Xen Orchestra project will be presenting at Xen Project User Summit too. If you've ever ... Read More
Tuesday, 26 August 2014 20:06
Tobias Kreidl
I would like to see XenServer/XenDesktop embrace similar technology to the VMware Project Fargo, where a clone on demand of a live... Read More
Saturday, 30 August 2014 15:00
Sam McLeod
Hi, will the talks be uploaded online after for those who can't attend? Thanks,
Thursday, 11 September 2014 04:13
Continue reading
11609 Hits
5 Comments

Pushing XenServer limits with Creedence beta.2

Well folks, its that time once again; we've another XenServer build ripe for your inspection, and this one is a critical one for a number of reasons. Today we've released XenServer Creedence beta.2, and this is binary compatible with a Citrix Tech Preview refresh. The build number is 87850 and it presents itself to the outside world as 6.4.95. Over the past few announcements I've hinted at pushing the boundaries of XenServer and wanting the community at large to "have at it", but I've not put out too many details on the overall performance we're seeing internally. The most important attribute of this build is that internally, its going to form part of a series of long term stability tests. Yes folks, we're that confident in what we're seeing and I wanted to thank everyone who has participated in our pre-release activities by sharing a few performance tidbits:

  • Creedence can start and run 1000 PV VMs with only 8GB dom0 memory. That's up from the 650 we have in XenServer 6.2.
  • Booting 125 Windows 7 VMs on a single host takes only 350 seconds in a bootstorm scenario. That's down from 850 seconds in XenServer 6.2
  • Aggregate disk throughput has been measured to improve by as much as 100% when compared to XenServer 6.2
  • Aggregate intrahost network throughput has been measured to improve by as much as 200% when compared to XenServer 6.2
  • The number of virtual disks per host has been raised by a factor of four when compared to XenServer 6.2

When compared to beta.1, the team has been looking at a number of performance and scalability system aspects, with a primary focus on dom0 idle state behavior at scale. This is a very important aspect of system operation as overall system responsiveness is directly tied to the overhead of managing a large number of VMs. We did see two distinct areas for investigation, and are inviting the community to look into these and provide us with others. Those two areas are:

  • When using 40Gb NICs outbound (transmit) performance is below expectations. We have some internal fixes, but are encouraging anyone with such NICs to test and report their findings
  • When large numbers of hosts are pooled we're seeing VM start times appear to slow unexpectedly under large pool VM densities.

 

As always we're actively encouraging you to test the beta and provide your feedback (both positive and negative) in an incident report. You can download beta.2 from here: http://xenserver.org/component/content/article/11-product/142-download-pre-release.html, and enter your feedback at https://bugs.xenserver.org.     

Recent Comments
Tim Mackey
@bpbp If you're comparing the ovs in Creedence to previous Creedence builds, then they are identical. If you're comparing it to ... Read More
Sunday, 24 August 2014 23:37
Tim Mackey
@vati, If I understand correctly, you're looking for when the DVSC will appear? The DVSC is being handled via the Citrix Tech Pr... Read More
Monday, 25 August 2014 14:32
Tim Mackey
@bpbp, We're hoping that the upgrade to ovs 2.1.2 will address most of the issues seen with the older ovs. If you're in a positi... Read More
Monday, 25 August 2014 14:33
Continue reading
18163 Hits
19 Comments

XenServer Creedence Reaches Beta Stage

In May we announced to the world that the next version of XenServer would be code named Creedence, and officially opened a public alpha program around this new version of XenServer. That alpha program was more successful than I'd expected with well over a thousand participants. Drawing these participants to Creedence was a combination of enthusiasm for XenServer as a virtualization platform, and a desire to see us make significant improvements in that platform. What greeted them was a full platform refresh including a 64 bit dom0, modern Linux kernel and the most recent Xen Project hypervisor. Over the following weeks we invited this enthusiast community to test three additional builds, each with increasing capabilities and performance.

As the audience for the XenServer Creedence message increased, we heard from some that they wanted to wait until we exited alpha mode and entered a beta phase. I'm happy to report that we've done just that with the release on August 5th, 2014 of beta.1 for XenServer Creedence. This means we're largely feature complete, and are looking seriously at the overall performance and stability of the platform. It also means we want you to start stressing the platform with your real-world workloads. Everything is fair game at this point, and we want to know where the breaking points are. Over the coming weeks you'll see blog posts covering some of the key performance and scalability improvements that we've achieved internally, but that's internally. Your experiences will vary, and we want to know about them.

We want you to push XenServer and tell us where your expectations weren't met. Here's how to do just that:

  1. Download beta.1 and install it on your favorite hardware. If it doesn't install, we want to know. If it didn't detect the devices you have, we want to know. Oddly enough we also want to know if it does!!
  2. Create any number of VMs using your preferred operating systems, or alternatively use your favorite provisioning solution and provision some VMs. If something goes wrong here, let us know.
  3. Install into those VMs the applications you care about. Again if something goes wrong here, let us know.
  4. Exercise your applications, verify if the performance you see is what you'd expect. If it isn't let us know.
  5. While your applications are running, test core virtualization functions like live migration, storage migration, high availability, etc. Everything is fair game. If something doesn't behave as you expect, let us know.

How do you let us know of issues, you ask? Simply create a new account at https://bugs.xenserver.org and report an incident (or incidents). It's that simple. If you're looking to discuss at a deeper technical level why something might be behaving a certain way, or are seeking debugging help, then our developer list might be helpful. You can subscribe to it at: https://lists.xenserver.org/sympa/info/xs-devel, but please understand that the developers are working hard to build XenServer and aren't product support folks.

 

We want Creedence to be the best XenServer release ever, and with your input it can be.     

Recent Comments
Tim Mackey
James, Thanks for the words of encouragement. beta.1 *should* be able to successfully upgrade 6.2 SP1 pools, and if it doesn't w... Read More
Wednesday, 06 August 2014 17:30
Bob Martens
Are we able to import raw Xen images?
Thursday, 07 August 2014 00:19
Tobias Kreidl
No Ceph support in Creedence, unfortunately. Keep lobbying for it! See: http://xenserver.org/discuss-virtualization/virtualization... Read More
Wednesday, 13 August 2014 02:54
Continue reading
14931 Hits
13 Comments

Debian 7.4 and 7.6 Guest VMs

"Four Debians, Two XenServers"

The purpose of this article is to discuss my own success with virtualizing "four" releases of Debian (7.4/7.6; 32-bit/64-bit) in my own test labs.

For more information about Debian, head on over to Debian.org - specifically here to download the 7.6 ISO of your choice ( I used both the full DVD install ISO as well as the net install ISO ).

Note: If you are utilizing the Debian 7.4 net install ISO the OS will be updated to 7.6 during the install process.  This is just a "heads up" in the event you are keen to stick with a vanilla Debian 7.4 VM for test purposes.  And so you will need to download the full install DVD for the 7.4 32-bit/64-bit release instead of the net install ISO.

Getting A New VM Started

Once I had the install media of my choice, I copied it to my ISO repository that both XenServer 6.2 and Creedence utilize in my test environment.

From XenCenter (distributed with Creedence Alpha 4) I selected "New VM".

In both 6.2 and Creedence I chose the "Debian 7.0 (Wheezy) 64-bit" VM template:

I then continued through the "New VM" wizard: specifying processors, RAM, networking, and so forth.  On the last step, I made sure as to select "Start the new VM Automatically" before I pressed "Create Now":

Within a few moments, this familiar view appeared in the console:

I installed a minimum instance of both: SSH and BASE system.  I also used guided partitioning just because I was in quite a hurry.

After championing my way through the installer, as expected, Debian 7.4 and 7.6 both prompted that I reboot:

Since this is a PV install, I have access to the Shutdown, Reboot, and Suspend buttons, but I was curious about tools as memory consumption, etc were not present under each guest's "Performance" tab:

... and the "Network" tab stated "Unknown":

Before I logged in as root - in both XenServer 6.2 and Creedence Alpha 4 - I mounted the xs-tools.iso.  Once in with root access, I executed the following commands to install xs-tools for these guest VMs:


mkdir iso
mount /dev/xvdd/ iso/
cd iso/Linux/
./install.sh

The output was exactly the same in both VMs and naturally I selected "Y" to install the guest additions:

Detected `Debian GNU/Linux 7.6 (wheezy)' (debian version 7).

The following changes will be made to this Virtual Machine:
  * update arp_notify sysctl.conf.
  * packages to be installed/upgraded:
    - xe-guest-utilities_6.2.0-1137_amd64.deb

Continue? [y/n] y

Selecting previously unselected package xe-guest-utilities.
(Reading database ... 24502 files and directories currently installed.)
Unpacking xe-guest-utilities (from .../xe-guest-utilities_6.2.0-1137_amd64.deb) ...
Setting up xe-guest-utilities (6.2.0-1137) ...
Mounting xenfs on /proc/xen: OK
Detecting Linux distribution version: OK
Starting xe daemon:  OK

You should now reboot this Virtual Machine.

Following the installer's instructions, I rebooted the guest VMs accordingly.

Creedence Alpha 4 Results

As soon as the reboot was complete I was able to see each guest VM's memory performance as well as networking for both IPv4 and IPv6:

XenServer 6.2

With XenServer 6.2, I found that after installing the guest agent - under the "Network" tab - there still was no IPv4 information for my 64-bit Debian 7.4 and 7.6 guest VMs.  This does not apply to 32-Bit Debian 7.4 and 7.6 guest VMs as the tools installed just fine.

Then I thought about it and realized that by disabling IPv6, presto - the network information appeared for my IPv4 address.  To accomplish this, I edited the following file (as to avoid adjusting GRUB parameters):

/etc/sysctly.conf

And at the bottom of this file I added:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 net.ipv6.conf.eth0.disable_ipv6 = 1

After saving my changes, I rebooted and immediately was able to see my memory usage:

However... I still could not see my IPv4 address under the "Network" tab until I noticed the device ID of the network interface -- it was Device 1 (not 0):

I deleted this interface and re-added a new one from XenCenter.  Instantly, I could see my IPv4 address and the device ID for the network interface was back to 0:

And yes, I tested rebooting -- the address is still shown and memory usage is still measured.  In addition I did try to remove the flags to disable IPv6, but that resulted in seeing "UNKNOWN" - again - for 64-Bit Debian 7.4 and 7.6 guests.  That just means in XenServer 6.2 I have kept my changes in /etc/sysctl.conf as to ensure my 64-Bit Debian 7.4 and 7.6 hosts with XenTools' Guest Additions for Linux work just fine.

So, that's that -- something to experiment and test with: Debian 7.4 or 7.6 32-bit/64-bit in a XenServer 6.2 or Creedence Alpha test environment!

 

--jkbs

@xenfomation

Recent comment in this post
JK Benedict
Tested on Creedence Beta, as well. Love it!!!
Thursday, 07 August 2014 18:58
Continue reading
19368 Hits
1 Comment

Beyond Creedence - XenServer 2015 Planning

In a few weeks James Bulpin and I will be at the Xen Project Developers Summit in Chicago, and some of our discussions will be about the future of XenServer, and more importantly to the community "What comes after Creedence?". With the Creedence alpha program we're seeing a level of community engagement which has honestly exceeded my expectations. I attribute this to the significant improvements in the platform, but also the level of transparency we've had with respect to early access to pre-release builds.

While it was pretty obvious what we needed to do to make Creedence viable, your input is important to the future success of XenServer. With that in mind, we'd like to hear what platform improvements you'd find most valuable. When I speak of platform improvements, I'm thinking of things like storage, networking, core virtualization, performance, scalability and operating system support. I'm not thinking of things which can be classified as data center or virtualization management, so things like network management, disaster recovery, or virtual machine provisioning are out of scope. Based on the blog comments for the various alpha announcements, we already know that CentOS 7 dom0, NFS4 and Ceph are on your wish lists, but what else?

Internally we use a "How would you spend $100?" model to prioritize changes, and if you were interested in providing feedback following that model, it would be ideal. If you've never used this model before, it's pretty simple. Write down the things you'd want to see (optionally with a "why" beside them), and then given a budget of $100. Spend the $100 by allocating it to your desired functionality, and anything with a zero is removed. This has the benefit of focusing on the high value changes without worrying about complexity. If you'd like to provide input, please do so in the comments section below, and let's see what the future of XenServer in 2015 looks like from your perspective.     

Recent Comments
David Reade
Could Changed Block Tracking (CBT) be considered as a feature please to speed up incremental backups? We use Unitrends to perform ... Read More
Wednesday, 30 July 2014 17:35
Keith Walker
CBT, the entire $100.
Tuesday, 31 March 2015 18:32
Andrew
$80 for CBT, $20 for online disk expansion.
Wednesday, 20 May 2015 13:12
Continue reading
56625 Hits
103 Comments

In-memory read caching for XenServer

Overview

In this blog post, I introduce a new feature of XenServer Creedence alpha.4, in-memory read caching, the technical details, the benefits it can provide, and how best to use it.

Technical Details

A common way of using XenServer is to have an OS image, which I will call the golden image, and many clones of this image, which I will call leaf images. XenServer implements cheap clones by linking images together in the form of a tree. When the VM accesses a sector in the disk, if a sector has been written into the leaf image, this data is retrieved from that image. Otherwise, the tree is traversed and data is retrieved from a parent image (in this case, the golden image). All writes go into the leaf image. Astute readers will notice that no writes ever hit the golden image. This has an important implication and allows read caching to be implemented.

tree.png

tapdisk is the storage component in dom0 which handles requests from VMs (see here for many more details). For safety reasons, tapdisk opens the underlying VHD files with the O_DIRECT flag. The O_DIRECT flag ensures that dom0's page cache is never used; i.e. all reads come directly from disk and all writes wait until the data has hit the disk (at least as far as the operating system can tell, the data may still be in a hardware buffer). This allows XenServer to be robust in the face of power failures or crashes. Picture a situation where a user saves a photo and the VM flushes the data to its virtual disk which tapdisk handles and writes to the physical disk. If this write goes into the page cache as a dirty page and then a power failure occurs, the contract between tapdisk and the VM is broken since data has been lost. Using the O_DIRECT flag allows this situation to be avoided and means that once tapdisk has handled a write for a VM, the data is actually on disk.

Because no data is ever written to the golden image, we don't need to maintain the safety property mentioned previously. For this reason, tapdisk can elide the O_DIRECT flag when opening a read-only image. This allows the operating system's page cache to be used which can improve performance in a number of ways:

  • The number of physical disk I/O operations is reduced (as a direct consequence of using a cache).
  • Latency is improved since the data path is shorter if data does not need to be read from disk.
  • Throughput is improved since the disk bottleneck is removed.

One of our goals for this feature was that it should have no drawbacks when enabled. An effect which we noticed initially was that data appeared to be read twice from disk which increases the number of I/O operations in the case where data is only read once from the VM. After a little debugging, we found that disabling O_DIRECT causes the kernel to automatically turn on readahead. Because data access pattern of a VM's disk tends to be quite random, this had a detrimental effect on the overall number of read operations. To fix this, we made use of a POSIX feature, posix_fadvise, which allows an application to inform the kernel how it plans to use a file. In this case, tapdisk tells the kernel that access will be random using the POSIX_FADV_RANDOM flag. The kernel responds to this by disabling readahead, and the number of read operations drops to the expected value (the same as when O_DIRECT is enabled).

Administration

Because of difficulties maintaining cache consistency across multiple hosts in a pool for storage operations, read caching can only be used with file-based SRs; i.e. EXT and NFS SRs. For these SRs, it is enabled by default. There shouldn't be any performance problems associated with this; however, if necessary, it is possible to disable read caching for an SR:

xe sr-param-set uuid=<UUID> other-config:o_direct=true

You may wonder how read caching differs from IntelliCache. The major difference is that IntelliCache works by caching reads from the network onto a local disk while in-memory read caching caches reads from either into memory. The advantage of in-memory read caching is that memory is still an order of magnitude faster than an SSD so performance in bootstorms and other heavy I/O situations should be improved. It is possible for them both to be enabled simultaneously; in this case reads from the network are cached by IntelliCache to a local disk and reads from that local disk are cached in memory with read caching. It is still advantageous to have IntelliCache turned on in this situation because the amount of available memory in dom0 may not be enough to cache the entire working set and reading the remainder from local storage is quicker than reading over the network. IntelliCache further reduces the load on shared storage when using VMs with disks that are not persistent across reboots by only writing to the local disk, not the shared storage.

Talking of available memory, XenServer admins should note that to make best use of read caching, the amount of dom0 memory may need to be increased. Ideally the amount of dom0 memory would be increased to the size of the golden image so that once cached, no more reads hit the disk. In case this is not possible, an approach to take would be to temporarily increase the amount of dom0 memory to the size of the golden image, boot up a VM and open the various applications typically used, determine how much dom0 memory is still free, and then reduce dom0's memory by this amount.

Performance Evaluation

Enough talk, let's see some graphs!

reads.png

In this first graph, we look at the number of bytes read over the network when booting a number of VMs on an NFS SR in parallel. Notice how without read caching, the number of bytes read scales proportionately with the number of VMs booted which checks out since each VM's reads go directly to the disk. When O_DIRECT is removed, the number of bytes read remains constant regardless of the number of VMs booted in parallel. Clearly the in-memory caching is working!

time.png

How does this translate to improvements in boot time? The short answer: see the graph! The longer answer is that it depends on many factors. In the graph, we can see that there is little difference in boot time when booting less than 4 VMs in parallel because the NFS server is able to handle that much traffic concurrently. As the number of VMs increases, the NFS server becomes saturated and the difference in boot time becomes dramatic. It is clear that for this setup, booting many VMs is I/O-limited so read caching makes a big difference. Finally, you may wonder why the boot time per VM increases slowly as the number of VMs increases when read caching is enabled. Since the disk is no longer a bottleneck, it appears that some other bottleneck has been revealed, probably CPU contention. In other words, we have transformed an I/O-limited bootstorm into a CPU-limited one! This improvement in boot times would be particularly useful for VDI deployments where booting many instances of the same VM is a frequent occurrence.

Conclusions

In this blog post, we've seen that in-memory read caching can improve performance in read I/O-limited situations substantially without requiring new hardware, compromising reliability, or requiring much in the way of administration.

As future work to improve in-memory read caching further, we'd like to remove the limitation that it can only use dom0's memory. Instead, we'd like to be able to use the host's entire free memory. This is far more flexible than the current implementation and would remove any need to tweak dom0's memory.

Credits

Thanks to Felipe Franciosi, Damir Derd, Thanos Makatos and Jonathan Davies for feedback and reviews.

Recent Comments
Tim Mackey
I'm not familiar with ZFS, but XenServer has had an shared storage cache called IntelliCache. It's designed for use in highly tem... Read More
Monday, 28 July 2014 02:19
Tobias Kreidl
Ross, Nice article! This cache is definitely going to help but as you pointed out, at some point, the size of the golden image wil... Read More
Tuesday, 29 July 2014 05:12
Tobias Kreidl
Apparently I hit a sore spot with you, "whatever"... I never said Nexenta was the best or most innovative solution out there, but... Read More
Thursday, 31 July 2014 04:28
Continue reading
40293 Hits
7 Comments

Running Scientific Linux Guest VMs on XenServer

Running Scientific Linux Guest VMs on XenServer

What is Scientific Linux?

In short, Scientific Linux is an customized RedHat/CentOS Linux distribution provided by CERN and Fermilab: popular in educational institutions as well as laboratory environments.  More can be read about Scientific Linux here: https://www.scientificlinux.org/

From my own long-term testing - before XenServer 6.2 and our pre-release/Alpha - Creedence - I have ran both Scientific Linux 5 and Scientific Linux 6 without issues.  This article's scope is to show how one can install Scientific Linux and, more specifically, ensure the XenTools Guest Additions for Linux are installed as these do not require any form of "Xen-ified" kernel.

XenServer and Creedence

The following are my own recommendations to run Scientific Linux in XenServer:

  1. I recommend using XenServer 6.1 through any of the Alpha releases due to improvements with XenTools
  2. I recommend using Scientific Linux 5 or Scientific Linux 6
  3. The XenServer VM Template one will need to use will either be of CentOS 5 or CentOS 6: 32 or 64 bit depends on the release of Scientific Linux you will be using

One will also require a URL as to install Scientific Linux from their repository, found at http://ftp.scientificlinux.org/linux/scientific/

The following are URLs I recommend for use during the Guest Installation process (discussed later):

Scientific Linux 5 or 6 Guest VM Installation

With XenCenter, the process of installing Scientific Linux 5.x or Scientific Linux 6 uses the same principles.  You need to create a new VM, select the appropriate CentOS template, and define the VM parameters for disk, RAM, and networking:

1.  In XenCenter, select "New VM":

2.  When prompted for the new VM Template, select the appropriate CentOS-based template (5 or 6, 32 or 64 bit):

3.  Follow the wizard to add processors, disc, and networking information

4.  From the console, follow the steps as to install Scientific Linux 5 or 6 based on your preferences.

5.  After rebooting, login as root and execute the following command within the Guest VM:

yum update

6.  Once yum has applied any updates, reboot the Scientific Linux 5 or 6 Guest VM by executing the following within the Guest VM:

reboot

7.  With the Guest VM back up, login as root and mount the xs-tools.iso within XenCenter:

7.  From the command line, execute the following commands to mount xs-tools.iso within the Guest VM as well as to run the install.sh utility:

cd ~
mkdir tools
mount /dev/xvdd tools/
cd tools/Linux/
./install.sh

8.  With Scientific Linux 5 you will be prompted to install the XenTools Guest Additions - select yes and when complete, reboot the VM:

reboot

9.  With Scientific Linux 6 you will notice the following output:

Fatal Error: Failed to determine Linux distribution and version.

10.  This is not a Fatal Error, but an error induced because the distro build and revision are not presented as expected.  This means that you will manually need to install the XenTools Guest Additions by executing the following commands and rebooting:

rpm -ivh xe-guest-utilities-xenstore-<version number here.x86_64.rpm
rpm -ivh xe-guest-utilities-<version number here>.x86_64.rpm
reboot

Finally after the last reboot (post guest addition install) one will notice from XenCenter that the network address, stats, and so forth are available (including the ability to migrate the VM):

 

I hope this article helps any of you out there and feedback is always welcomed!

--jkbs

@xenfomation

 

Recent Comments
Terry Wang
Running PV on HVM (also called PVHVM sometimes) is just fine. For modern Linux distros with Linux 3.0+ kernel (it'll unplug the QE... Read More
Monday, 28 July 2014 03:56
JK Benedict
Stay tuned! I have more to offer for Creedence... especially in lieu of Mr. Mackey's request from the following article @ http://... Read More
Saturday, 27 September 2014 09:03
Ian Yates
Hi, I'm new to this community but independently worked out a (pretty much identical) install routine for ScientificLinux on Xen so... Read More
Wednesday, 30 July 2014 10:24
Continue reading
20877 Hits
3 Comments

XenServer Creedence Tech Preview and Creedence Alpha

This morning astute followers of XenServer activity noticed that Citrix had made available the previously announced Tech Preview for Creedence. The natural follow on question is how this relates to the alpha program we've been running on xenserver.org. The easy answer is that the Citrix Tech Preview of XenServer Creedence is binary compatible with the alpha.4 pre-release binary you can get from xenserver.org. From the perspective of the core platform (i.e. XenServer virtualization bits), the only difference is in the EULA.

So why run a Tech Preview if you have a successful alpha program already?

That's where the differences between a pure open source effort and a commercial effort begin. While the XenServer platform components are binary compatible, Citrix customers have expectations for features which couldn't be made open source, or implementations directly supporting Citrix commercial products. Perfect examples of these features and implementations can be seen on the Tech Preview download page, such as Microsoft System Center Integration, expanded vGPU support for XenDesktop, and the return of both the DVSC and WLB. While there is no guarantee any of those features or specific implementations will be present in the final Citrix release, or for that matter under what license, Citrix is seeking your input on them and a Tech Preview program is how that is accomplished.

I can't access the Tech Preview site; what's wrong?

If you can't login to the Tech Preview site, that likely means your account isn't associated with a Citrix commercial contract for XenServer. Since the alpha.4 pre-release is binary compatible, you can experience all the platform improvements yourself by downloading XenServer from the xenserver.org pre-release download page.

How do I provide feedback?

If you are able to participate in the Tech Preview program, you'll find the options for feedback listed on the Tech Preview page. Of course, even if you can participate in the Tech Preview program we're always accepting XenServer feedback through our public feedback options:

The XenServer incident database: https://bugs.xenserver.org

Development feedback (xs-devel): https://lists.xenserver.org/sympa/info/xs-devel

What cool things are in alpha.4?

This is the best for last! We've already got in Creedence a 64 bit dom0, updated Linux 3.10 kernel, updated open virtual switch 2.1.2, improved boot storm handling, read caching for file based SRs; so what goodies are in here for the core platform people? Let's start with TRIM and UNMAP to better reclaim storage, then add in 32 bit to 64 bit VM migration to support upgrade scenarios and storage migration from XenServer 6.2 and prior, all with additional operating system validation with SLES 11 SP3 and Ubuntu 14.04 LTS.

What testing would you like us to do?

Don't let the alpha label fool you, alpha.4 of XenServer Creedence has been through quite a bit of testing, and is very much ready for you to try and stress it. The one thing we can say is that we're still working on the performance tuning so if you push things really hard, dom0 may run out of memory and you might need to follow CTX134951 to increase it (valid values are 8192, 16384 and 32768). This is particularly true if you're running more than 200 VMs per host, or need to attach more than 1200 virtual disks to VMs.     

Recent Comments
Tim Mackey
James, We now have the code in place to do a 32 bit to 64 bit migration, so upgrades are now on the table for testing, but I woul... Read More
Thursday, 10 July 2014 20:44
valentin radu
Hello, You can tell me when it will be released new stable version? Thx,
Tuesday, 15 July 2014 16:29
Hong Lae Cho
Hey Tim, Would you be able to provide a bit more clarification regarding "UNMAP" functionality in Creedence Alpha 4 & Tech Previ... Read More
Monday, 21 July 2014 05:10
Continue reading
25899 Hits
9 Comments

About XenServer

XenServer is the leading open source virtualization platform, powered by the Xen Project hypervisor and the XAPI toolstack. It is used in the world's largest clouds and enterprises.
 
Technical support for XenServer is available from Citrix.