Virtualization Blog

Discussions and observations on virtualization.

What is Citrix Doing with and Why


This article provides additional information regarding the upcoming rebranding of "XenServer" to “Citrix Hypervisor” and what steps Citrix is taking in conjunction with this change.

As many of you know, was created many years ago to provide developers and users of the Free edition a venue through which to share insights and suggestions regarding product features, enhancements, and technical support. The upcoming rebranding of "XenServer" to "Citrix Hypervisor" offers an opportunity to consolidate access to product resources to a single website - To be clear, this does not mean developers and users of the Free edition will cease to have access to the resources they need to simplify, enhance, and extend their virtual environments. On the contrary, such resources will continue to be available on the Citrix Hypervisor pages of and

The primary reason for consolidating access to product resources to is simple - to encourage developers and users to visit the same website for Citrix Hypervisor resources as they do for all other Citrix products and solutions. Whether it be product documentation, software components, developer tools, technical support, blog articles, case studies, etc., we believe taking this step will help us promote Citrix Hypervisor as an invaluable component of a complete Citrix virtualization stack that enables our customers to get the most out of their virtual applications and desktops environments.

Directing visitors to to access all things Citrix hypervisor-related will help us convey the message that all products and solutions developed by Citrix, along with their respective components and resources, are available at, and must be accessed from,

Citrix readily acknowledges and appreciates the input and feedback we receive from developers and users of all product editions. Thus, we cannot emphasize enough the only change visitors of will notice starting April 1st is where they go to access Citrix Hypervisor components and resources. Users of all product editions, along with members of the development community, will retain the opportunity to share their input and suggestions with Citrix to ensure the features and capabilities we introduce in future releases enable our customers to get the most out of their virtual environments.



Citrix Hypervisor PM



Continue reading
4659 Hits

Citrix Releases Cumulative Update 2 for XenServer 7.1 LTSR

Citrix is pleased to announce the release of the second cumulative update (CU2) for XenServer 7.1. This update, which Citrix recommends applying at your earliest convenience, is strictly for customers running XenServer 7.1 LTSR. As with previous cumulative updates, this update includes fixes only, no new features are introduced with this update.

To download XenServer 7.1 CU2, which can be installed using the full installation ISO or the update ISO, visit and log in with your Citrix account.

Citrix recommends reading the XenServer 7.1 CU2 Release Notes prior to installing the update.


Continue reading
2912 Hits

XenServer 7.6 Now Available for Download!

Hello everyone,

Citrix is pleased to announce general availability of XenServer 7.6, the latest release for those following the Current Release model. For information regarding the differences between the Long-Term Service Release (LTSR) and Current Release (CR) models offered by Citrix, please refer to the article published on the Citrix blog site.

7.6 introduces official support for thin provisioning on block storage for XenServer virtual disk Storage Repositories along with Networking SR-IOV for enhanced system I/O performance. Initially available as “experimental” features in the 7.5 CR release, rapid and wide implementation of these features in production environments is anticipated as customers look to achieve significant savings in storage cost and capacity as well as better overall system performance.

If it's been some time since your organization has upgraded its XenServer virtual environment, please review the following list, which highlights the various features introduced in recent versions (prior to 7.6) that are helping our customers get the most out of their virtual workloads.

7.5 CR:

  • Larger XenServer Resource Pool size: XenServer now supports up to 64 hosts participating in a resource pool
  • Support for USB Pass-Through, enabling physical USB devices to be passed through to a VM so the guest OS can use the physical USB device as a local USB device

7.1 CU1 LTSR:

  • Automated updates to simplify host maintenance
  • Delivering Windows VM drivers via Windows updates (WSUS)
  • Health Check (via Citrix Insight Services) to monitor the health of XenServer hosts
  • Live-Patching, enabling hosts to be updated (patched) without rebooting to minimize downtime
  • PVS-Accelerator -reducing/eliminating boot storms while dramatically improving virtual desktop performance
  • SMB storage, providing even more options for SRs
  • Bitdefender Hypervisor Introspection (HVI), which scans raw memory at the hypervisor later to detect and protect virtual environments from sophisticated IT attacks (i.e., viruses, malware, ransomeware and rootkit exploits)
  • Support for Nutanix, combining the benefits of a single software stack (XA/XD/XS) with the cost-effective scale and management benefits provided by hyper-converged infrastructure

Consistent with previous releases, XenServer 7.6 delivers new features to help our customers get the most out of Citrix Virtual Apps and Desktops.

For additional information pertaining to XenServer, which, along with many other Citrix products, is currently going through the rebranding process (and will soon be referred to as “Citrix Hypervisor”), please visit the XenServer/Citrix Hypervisor page on

Until next time,



Recent Comments
At some point will it launch 7.1 CUXX with the new features implemented until now (7.2 - 7.6) or do we have to wait for one in the... Read More
Saturday, 08 September 2018 18:30
Andy Melmed
Hi Daniel, Any new features introduced between 7.2 and 7.6 CR will not be back-ported to 7.1 CUxx LTSR. CUxx is a roll-up of bug ... Read More
Thursday, 04 October 2018 00:48
Rafique Uddin
XENSERVER 7.6 this the free version. Do we need license for this ?
Thursday, 04 October 2018 00:06
Continue reading
14135 Hits

Citrix Announces General Availability of XenServer 7.5 CR

Hello everyone,

This week Citrix released the latest version of XenServer: 7.5 CR. For those unfamiliar with the LTSR and CR release models offered by Citrix, please refer to the recent article published on the Citrix blogs that explains the difference between the two release models.

So what’s new in 7.5 you may be asking? Great question!

Before I answer this question however, I’d like to remind everyone of the many features we’ve introduced since the release of 7.1 (LTSR) back in February 2017, starting with 7.1 itself.

7.1 LTSR features:

  • Automated updates to simplify host maintenance
  • Delivering Windows VM drivers via Windows updates (WSUS)
  • Health Check (via Citrix Insight Services) to monitor the health of XenServer hosts
  • Live-Patching, enabling hosts to be updated (patched) without rebooting to minimize downtime
  • PVS-Accelerator -reducing/eliminating boot storms while dramatically improving virtual desktop performance
  • SMB storage, providing even more options for SRs
  • Bitdefender Hypervisor Introspection (HVI), which scans raw memory at the hypervisor later to detect and protect virtual environments from sophisticated IT attacks (i.e., viruses, malware, ransomeware and rootkit exploits)
  • Support for Nutanix, combining the benefits of a single software stack (XA/XD/XS) with the cost-effective scale and management benefits provided by hyper-converged infrastructure

7.2 CR features:

  • Windows Continuum (tablet mode enablement) – providing seamless transition from desktop to tablet mode on virtual desktops
  • Scheduled snapshots for greater control and more flexible backup capabilities

7.3 CR features:

  • Citrix Director integration to simplify troubleshooting of virtual desktop issues
  • Bromium Secure Platform - providing advanced protection for Windows VMs
  • XenCenter host status and licensing visibility
  • Expanded Linux template support
  • VLAN tagging for management and storage interfaces to improve network utilization

With the release of 7.4 CR, XenServer became the first hypervisor to support vGPU-accelerated VM Live-Migration (XenMotion)! In addition, 7.4 introduced support for AMD MxGPU, further extending its support for the major graphics vendors (NVIDIA, Intel, AMD). Combined, these features reinforced XenServer’s lead in advanced virtual graphics capabilities.

Now that we’ve reviewed the many features that have been introduced since 7.1, let’s highlight what’s new in 7.5.

First and foremost, 7.5 introduces support for GFS2. With GFS2 support comes several experimental features, including thin provisioning on block storage for virtual disk storage repositories, as well as a significant increase in virtual disk size: from 2TBs to 16TBs. Yes, that’s correct… an 8x increase in storage capacity to help our customers with their ever-increasing data requirements!

With newly added support for Networking SR-IOV: Pass-Through of Virtual Functions, XenServer can now use Single Root I/O Virtualization that allows a single PCI device to appear as multiple PCI devices on the physical system. By assigning one or more VFs to a VM, the guest OS can use the device(s) as if it was directly assigned.

For large-scale virtual environments, customers can now take advantage of the significant increase in the number of supported hosts per XenPool – from the current limit of 16 to as many as 64.

7.5 introduces support for USB Pass-Through, enabling physical USB devices to be passed through to a VM so the guest OS can use the physical USB device as a local USB device.

Lastly, several changes have been made in 7.5 pertaining to guest OS support, including the addition of the following Linux guest OS templates:

  • CentOS 7.4
  • CentOS 6.9
  • Oracle Linux 6.9
  • Red Hat Enterprise Linux 6.9
  • Scientific Linux 6.9

Please note the majority of the features described above are available in XenServer 7.5 Enterprise edition only. For more information regarding the newest features in 7.5, refer to the Release Notes.

As you can see, XenServer continues to push the envelope when it comes to ease-of-use, security, graphics, performance, and scalability. And there are a lot more great things coming your way in future releases!

Until next time,



Recent Comments
Martin Cerveny
vgpu-*.src.rpm is missing in XenServer-7.5.0-source.iso. Where to find it ?
Saturday, 02 June 2018 08:47
Andy Melmed
Hi Martin, Thank you for your inquiry. As we received several inquiries asking why the vgpu-*.src.rpm is missing from the 7.5 so... Read More
Thursday, 07 June 2018 15:05
Continue reading
8911 Hits

XenServer 7.4 Available for Download!

Hello XenServer community,

It is our pleasure to announce the release of 7.4, the latest update in the CR release track.

XenServer 7.4 offers customers following the CR release track access to the latest product features designed to enhance application, desktop and server virtualization deployments. For customers running XenServer 7.3 (up until now the most recent CR), Citrix recommends upgrading to 7.4 at the earliest convenience to maintain support.

For customers following the LTSR release track (starting with XenServer 7.1 LTSR), Citrix recommends reviewing the XenServer 7.4 release notes to determine if 7.4 would deliver additional benefits to their virtual environments.

Note: The decision to embark on the CR release track entails the adoption of new CRs as they are made available in order to maintain support, as hotfixes for each CR are only made available for up to 4 months after the next CR is released. Please see the XenServer Product Lifecycle page for details.

XenServer 7.4 is available in the following editions:

  • Free Edition
  • Standard Edition
  • Enterprise Edition


The release of 7.3 introduced several changes to feature availability in the Free Edition of XenServer. For details regarding these changes, please see XenServer 7.3: Changes to the Free Edition.


The following features and functionality are new in XenServer 7.4:

  • vGPU-enabled VM live-migration (XenMotion)
  • AMD MxGPU support
  • Citrix Cloud XenApp/XenDesktop subscriptions to XenServer (licensing)

For more information, including the edition(s) in which each feature can be found, refer to the XenServer 7.4 release notes.

To download XenServer 7.4, please visit the XenServer Product Download page.




Recent Comments
Ilsa Loving
Why is 7.1 no longer available? I thought it was supposed to be an LTSR version. While the removal of features was dissappointin... Read More
Wednesday, 07 March 2018 01:03
David Cottingham
Hi Ilsa, Back in August last year Citrix announced that hotfixes on each release would be available to Free Edition customers unt... Read More
Thursday, 08 March 2018 01:26
Andreas Becker
Where are the nVidia Grid drivers for XenServer 7.4 ??? Can´t find them at nVidia Enterprise Licensing Portal ...
Thursday, 15 March 2018 17:35
Continue reading
18736 Hits

Announcing the Availability of XenServer 7.3

We're pleased to announce the availability of XenServer 7.3, the latest Current Release (CR)!

CRs Versus LTSRs

XenServer 7.3 offers customers following the CR release track access to the latest product features designed to enhance application, desktop and server virtualization deployments. For customers running XenServer 7.2 (up until now the most recent CR), Citrix recommends upgrading to 7.3 at the earliest convenience to maintain support.

For customers following the LTSR release track (starting with XenServer 7.1 LTSR), Citrix recommends reviewing the XenServer 7.3 release notes to determine if 7.3 would deliver additional benefits to their virtual environments.

Note: The decision to embark on the CR release track entails the adoption of new CRs as they are made available in order to maintain support, as hotfixes for each CR are only made available for up to 4 months after the next CR is released. See the XenServer Product Lifecycle page for details.

XenServer 7.3 is available in the usual three editions:

  • Free Edition
  • Standard Edition
  • Enterprise Edition

Important Notice for XenServer Free Edition Users

With the release of 7.3 comes several changes to feature availability in the Free Edition of XenServer. For details regarding these changes, please see XenServer 7.3: Changes to the Free Edition.

New Features in XenServer 7.3

The following is a list of the features and functionality introduced in XenServer 7.3:

  • Enhanced security for Windows virtual machines by support for Bromium Secure Platform
  • Changed Block Tracking API (for enhanced backups)
  • New and expanded guest OS support
  • Support for VLAN tagging on management and storage network interfaces
  • IGMP snooping for IPv4 multicast support
  • SMB v3.0 for ISO storage repositories (SRs)
  • Improvements to XenServer Conversion Manager
  • Localization for Email Performance Alerts
  • Support for BIOS Asset Tags
  • Citrix Director integration
  • Enablement for XenDesktop Tablet Mode (enables Windows 10 Continuum experience in virtualized desktop environments)

For more information, including the edition(s) in which each feature can be found, please refer to the XenServer 7.3 release notes.

To download XenServer 7.3, please visit the XenServer Product Download page.




Continue reading
11023 Hits

XenServer 7.3: Changes to the Free Edition

Today we're pleased to announce the release of XenServer 7.3, which includes a raft of new features and improvements. Whilst you can read all about exactly what's in that release over on the release announcement blog, I wanted to provide clarity on the changes we're making to the Free Edition of XenServer as part of the release.

As many of you will be aware, XenServer is available in three editions: Free, Standard, and Enterprise. All of them are installed from the same ISO, and all are equally open source. The Standard edition is almost identical to the Free edition in terms of feature set, but includes commercial support and an extended period of hotfix availability from Citrix, whilst the Enterprise edition enables a variety of additional features.

Having carefully considered what features are in each edition, we've taken the decision to move some features out of the Free edition, and into Standard.

The full list of features being moved is:

  • Dynamic Memory Control
  • Xen Storage Motion
  • Active Directory Integration
  • Role Based Access Control
  • High Availability
  • GPU Pass-Through
  • Site Recovery Manager (Disaster Recovery)
  • XenCenter Rolling Pool Upgrade Wizard
  • Maximum Pool Size Restricted To 3 Hosts (existing larger pools will continue to work, but no new host joins will be permitted)

This will therefore make the Standard edition substantially different to the Free edition. No features are moving from Free to Enterprise. You can find all of the details in the full XenServer 7.3 feature matrix.

I suspect many readers will be asking "why?". We have thousands of customers who trust XenServer to host their workloads, and we need to make sure we can invest in the product for them. Of the many thousands of customers using the Free edition, we hope that those using it for large deployments (and likely to be using the features above) will consider purchasing a subscription to enable access not just to the features above, but also the patch/hotfix stream for each release for an extended period (roughly 7 months from release versus 3 months for Free users), access to Long Term Service Releases (up to 10 years of maintenance) and support services from Citrix.

I realise that this news will be difficult for people who make use of the Free edition of XenServer in larger environments. We have looked carefully at how the revised Free edition compares to other free virtualisation platforms on the market, and concluded that even with this change, XenServer's free feature set is still great for small deployments or home labs. Evidently Citrix will continue to add functionality to all three editions in the future: as you can see from 7.3 release notes, there is plenty going on!

As usual, you can get in touch with me via the comments function below, or on Twitter. I'll endeavour to reply as soon as practical!

Recent Comments
Sam McLeod
David, you know I hold the upmost respect for you so please don't take this personally. This is a typical, big enterprise, Micros... Read More
Thursday, 14 December 2017 02:03
David Cottingham
Hi Sam, Firstly, I'll say that I can understand where you're coming from. I recognise that there will be some people affected by ... Read More
Thursday, 14 December 2017 03:32
Sam McLeod
What happens if a host dies in one of your existing pools and you need to replace it? Unless your pool is tiny (3 or less hosts) -... Read More
Thursday, 14 December 2017 05:05
Continue reading
49195 Hits

A Great Time to Take A Look at XenServer Enterprise!

Good afternoon everyone,

As we make our way through the last quarter of the year, I wanted to remind the community of the significant progress the XenServer team has achieved over the last 18 months to make XenServer the awesome hypervisor that it is today!

While many of you have been making the most of your free XenServer hypervisor, I would like to take this opportunity to review just a few of the new features introduced in the latest releases of the Enterprise edition - features that our customers have been using to optimize their application and desktop virtualization deployments.

For starters, we've instrumented automated updates and live patching, features that streamline the platform upgrade process by enabling multiple fixes to be installed and applied with a single reboot and in many cases, no reboot whatsoever, significantly reducing downtime for environments that require continuous uptime.

We've also worked with one of our partners to introduce a revolutionary approach to securing virtual workloads, one that is capable of scanning raw memory at the hypervisor layer to detect, protect and remediate against the most sophisticated attacks on an IT environment. This unique approach provides an effective line of defense against viruses, malware, ransomware and even root kit exploits. What's more, this advanced security technique complements security mechanisms already implemented to further strengthen protection of critical IT environments.

Providing a local caching mechanism within the XenServer hypervisor enables our virtual desktop customers to dramatically improve the performance of their virtual desktops, particularly during boot storms. By caching requests for OS image contents in local resources (i.e., memory and storage), XenServer is able to work with Provisioning Services to stream contents directly to virtual desktops, reducing resource utilization (network and CPU) while enhancing user productivity.

Expanded support for virtual graphics allows our customers to leverage their investments in hardware from the major graphics vendors and enable GPU-accelerated virtual desktops that effectively support graphics-intensive workloads.

Designing, developing and delivering features that bring out the best in virtualization technologies... that's our focus. And thanks to the invaluable insight and feedback provided by this community, will continue to be the driving force behind our innovation efforts.

Interested in evaluating the features described above, click here.

Until next time,



Continue reading
3682 Hits

Introduction to Saving VM Parameters: How I Metadata Backup

Introduction to Saving VM Parameters: How I Metadata Backup

XenServer Virtual Machines (VMs) certainly need no introduction, but even if you do not pardon the pun above, they still contain a lot of specialized and individualized information about their sizes, network connections, and myriad other settings that are generally not readily exposed, yet are integral to the operation and functionality of each VM. This blog entry is not intended to take a deep dive into the several hundred parameters that are defined for VMs, but rather just talk a bit about how to save, extract and potentially restore VM information based on them.

VM Metadata Backups

The purpose of backing up the metadata from a VM is to help you understand how it’s configured without having to search through the list of parameters accessible via various “xe” commands or XenAPI calls that require some programming efforts, plus also allow you to potentially track changes to your VMs without necessitating a full VM export/backup each time. You may not want or need to restore a VM from a full backup, but rather just revert a few parameters back to older values. You might also want to monitor what sorts of changes have taken place over time and equate those to performance or other metrics. In short, a number of reasons to maintain relatively frequent metadata backups of VMs can be justified.

Getting VM Metadata

There are a number of ways to obtain VM metadata settings. One of these is with the standard “xe” command to extract parameters, either individually, in a comma-separated string of multiple queries, or all of them such as with:

# xe vm-list uuid|name-label=UUID|NAME-LABEL params=all

Here, either the UUID or name-label can be used to select the VM.

Some parameters can also not only be obtained via “xe vm-param-get” but also changed, using the complementary “xe vm-param-set” operator. This gives you access to modifying around 30 parameters and reading over 80 of them.

The XenCenter console, xsconsole, provides a direct way to back up and restore VM metadata. From the "Backup, Restore and Update" menu, you can next navigate to the "Backup Virtual Machine Metadata" option and choose from available SRs onto which you wish to create a metadata backup of all available VMs. Note, though, that it will only evidently create metadata backups of running VMs! Likewise, the restore operation can only be performed on various subsets of VMs and not on an individual VM. Conversely, the metadata restore operation apparently can be applied to both running and halted VMs, but again cannot be performed on an individual VM.

Another option is to make use of the XenAPI library and extract tokens using XenAPI calls and access them for example using constructs such as these:

    vm_meta_status = gather_vm_meta(vm_object, full_backup_dir)

    vm_record = session.xenapi.VM.get_record(vm_object)

    vm_out = open ('%s/vm.cfg' % tmp_full_backup_dir, 'w')

    vm_out.write('name_label=%s\n' % vm_record['name_label'])

    vm_out.write('name_description=%s\n' % vm_record['name_description'])

    vm_out.write('memory_dynamic_max=%s\n' % vm_record['memory_dynamic_max'])

    vm_out.write('VCPUs_max=%s\n' % vm_record['VCPUs_max'])

    vm_out.write('VCPUs_at_startup=%s\n' % vm_record['VCPUs_at_startup'])


This can be time-consuming if you want to keep identifying and modifying code to deal with any additions or changes, plus you may periodically have to update your API libraries.

Yet another option is to make use of the not-well-documented features within the “xe” command set associated with vm-export and vm-import utilities. It is possible to export just the metadata from a VM using the following syntax:

# xe vm-export metadata=true uuid=UUID-OF-VM) filename=/full_path/OUTPUT_FILE.XVA


This will create what is in essence a tar file containing a single file that captures more than 300 parameters! The XML code has to be extracted from this tarball, which contains a single file that is always named ova.xml and can be pulled from the XVA files with a basic tar command in which specifying the file to extract as ova.xml is optional, since it’s the one and only file within the tar file:



tar: ova.xml: implausibly old time stamp 1969-12-31 17:00:00


Note that you may get this rather interesting message regarding the timestamp, which can be ignored.  It may also turn out that the output file has absolutely no access permissions set, so you may want to run a “chmod 600 ova.xml” (or 644, etc.) to make it readable. You may also wish to rename it so it’s unambiguous and/or less likely to be overwritten.

For exported XVA files that are gzipped, you can extract the ova.xml file in a single operation with:

# tar xzf OUTPUT_FULE.XVA .gz

tar: ova.xml: implausibly old time stamp 1969-12-31 17:00:00

Once extracted, let’s take a look at the first part of the ova.xml file, which takes on a rather “ugly” appearance:


There is a wealth of information in here, but it’s not in a very friendly format. Fortunately, this can be readily rectified with the handy xmllint utility already present on XenServer (at least on 7.X):

# tar -xOf /ubuntu12-xs66-specialchars.XVA |  xmllint –format - >/tmp/output_VM.xml

The ”-O” flag causes the output to be redirected to stdout and hence it can be piped to the xmllint utility, which in turn can generate a very nicely formatted and properly indented XML file. Note that the “-“ before the redirection “>” operator signifies the output of xmllint to go to stdout and if desired, the command can be abbreviated as such, in which case the output will just appear on the terminal. It will be several hundred lines long, so you may as well redirect the output into a file where you can more conveniently deal with reviewing that amount of information.

Here is what the first part of the formatted file really looks like: 


OK, Great -- Now What?

Given the ability to now parse and peruse the XML metadata file associated with a particular VM, one could contemplate creating periodic backups of the VM metadata to have on hand in case one needs to reconstruct something or check if anything had changed. That’s all fine and good, but other than using “xe” commands or other means to change individual parameters, how does having these data help in the event of wanting to reconstruct or restore a VM?

The bottom line is that this feature has limited direct applications, though it does have a few. Consider the case of trying to use an XVA file that only contains the VM’s metadata to restore a VM. Note that the original VM must of course still exist or there will be nothing present to associate the VM storage with if a version of this VM cannot be found. However, if it is present, consider the following results:

# xe vm-import preserve=true filename=/ubuntu12-xs66-specialchars.XVA

The VM cannot be imported unforced because it is either the same version or an older version of an existing VM.

vm: 9538882a-c7e4-b8e5-c1f9-0d136f4a81b1 (TST-ubuntu12-vmtst3-xs66)

existing_version: 0

version_to_import: 0


Perhaps as anticipated, it will fail as the “preserve=true” flag will first check for a duplicate VM and upon discovering it, flags it as a command that would overwrite the existing VM. That’s a good thing. Leaving off that flag, we next try:

xe vm-import filename=/ubuntu12-xs66-specialchars.XVA metadata=true

This should yield success, but what kind of success? What happens is the VM created using the “metadata=true” flag produces a new VM copy with the same name, but a different UUID for the VM that is “Created by template provisioner.” What has happened is just that a fast clone has been created. You will see that if you delete such a VM created that way, it will not show any storage devices associated with it and in XenCenter, it will therefore not ask you if you want to delete the associated storage.

This is not entirely without use, however as you can still make use of this VM and perhaps even compare its characteristics to the original. Furthermore, you can export the VM, and import it as a new VM in which case it will gain the properties of a full clone. At that point, dependence on the original no longer exists.

This exercise might be useful in debugging or checking parameter-based performance or other issues between the original and subsequent metadata modifications. Such headers may also be useful just for tracking historical uses of VMs, checking to see what IP addresses may have been assigned, and numerous other things.

The Full XVA Export

The discussion up to this point should result in a mental lightbulb turning on and raising the question, well, if I restore a full export of a VM, isn’t all this information already in there? Since clearly it has to be for a vm-import to work properly, an examination of a full XVA export will indeed reveal that it consists of many fairly small files, numbering at times many thousands, but always starting with our old friend, ova.xml, as we see from this sample output that lists the contents instead of extracting it:

# tar -tvf  /exports/test-export.xva |less

---------- 0/0           29935 1969-12-31 17:00 ova.xml

---------- 0/0         1048576 1969-12-31 17:00 Ref:367/00000000

---------- 0/0              40 1969-12-31 17:00 Ref:367/00000000.checksum

---------- 0/0         1048576 1969-12-31 17:00 Ref:367/00000001

---------- 0/0              40 1969-12-31 17:00 Ref:367/00000001.checksum

---------- 0/0         1048576 1969-12-31 17:00 Ref:367/00000002

---------- 0/0              40 1969-12-31 17:00 Ref:367/00000002.checksum

---------- 0/0         1048576 1969-12-31 17:00 Ref:367/00000003

---------- 0/0              40 1969-12-31 17:00 Ref:367/00000003.checksum

---------- 0/0         1048576 1969-12-31 17:00 Ref:367/00000004

---------- 0/0              40 1969-12-31 17:00 Ref:367/00000004.checksum


The nice aspect of having the XVA file self-contained with all the metadata, as well as the data contents of the VM, makes this standalone file easy to move around and utilize as a backup.

The other nice aspect is that you, in fact, can use it as its own metadata storage mechanism and extract only that part of it if desired without needing to create a separate metadata backup (unless, of course, you want to do that more often and independently of a a vm-export). To extract just the metadata from this file shown above, all you need to do is to specify the embedded tar file name:

tar -xvf  /exports/test-export.xva ova.xml

We now have the same metadata file content we had when running the vm-export command combined with the “metadata=true” option.

In Summary

First off, it cannot be overstated that your XenServer environment should be backed up frequently and fastidiously, including both the pool metadata as well as the individual metadata for VMs. Even if you already have full exports of your VMs, having additional metadata can be useful for auditing purposes, as well as making it possible to check on parameters that are hard or impossible to glean through other means.

Nobody that I know was ever accused of creating too many backups.

Continue reading
7806 Hits

XSO DB for XS 7.1 Closing Down

Attention XenServer community:

Please be advised that, in alignment with our goal of frequent releases and the XSO database supporting one active release to help resolve issues found in that release while simultaneously preparing for the next release, we are closing down the XSO database for 7.1 issues. As such, users are encouraged to move to XS 7.2 and raise any issues they encounter with that release here. Users who wish to remain on the Long-Term Support Release (LTSR) stream are encouraged to visit

As always, thank you for your support.



Continue reading
3453 Hits

Citrix Announces First Cumulative Update for XenServer 7.1

Citrix has posted a blog announcing the availability of XenServer 7.1 CU1, the first cumulative update for the XenServer 7.1 Long-Term Support Release (LTSR).

CU1 rolls-up all hotfixes released on XenServer 7.1 to date. While there are no new features introduced in this update, a number of additional fixes are included to further enhance and simplify XenApp and XenDesktop environments. Information pertaining to these fixes can be found in the release notes.

Of particular significance, this cumulative update provides customers on Long-Term Support Release (LTSR) 7.1 with the latest platform fixes applicable to v7.1. Unlike Current Releases (CR), which provide access to new features on an almost quarterly basis, Long-Term Support Releases are designed for customers who, possessing the features they need to optimize their XenApp and XenDesktop environments, intend to remain on the LTSR for the foreseeable future. For these customers, using a Long-Term Support Release offers stability - Citrix will support the LTSR for up to 10 years – as well as access to fixes as they become available. You can learn more about the differences between Long-Term Support and Current Releases here.

Wondering if XenServer 7.1 Enterprise edition is right for your organization? Click here to download a free trial version.




Recent comment in this post
Sam McLeod
Great to see a roll-up for all hotfixes for 7.1 to date, it was very painful figuring out what patches to apply and what not to in... Read More
Sunday, 17 September 2017 23:15
Continue reading
4985 Hits
1 Comment

Changes to Hotfixes and Maintenance for XenServer Releases

Changes to Hotfixes and Maintenance for XenServer Releases

XenServer (XS) is changing as a product with more aggressive release cycles as well as the recent introduction of both Current Release (CR) and Long Term Service Release (LTSR) options. Citrix recently announced changes to the way XenServer updates (hotfixes) will be regulated and this article summarizes the way these will be dealt with looking forward.

To first define a few terms and clarify some points, the CR model is intended for those who always want to install the latest and greatest versions, and these are being targeted for roughly quarterly releases. The LTSR model is primarily intended for installations that are required to run with a fixed release. These are by definition required to be under a service contract in order to gain access to periodic updates, known as Cumulative Updates or CUs. The first of these will be CU1. Since Citrix is targeting support for LTSR versions for ten years, this incurs quite a bit of cost over time and is much longer than a general CR cycle is maintained, hence the requirement for active support. The support for those running XS as part of running licensed versions of XenApp/XenDesktop will remain the same, as these CR licenses are provided as part of the XenApp/XenDesktop purchase. There will still be a free version of XenServer and the only change at this point will be that on-demand hotfixes for a given CR will no longer be available once the next CR is released. Instead, updates to quarterly releases will be needed to access and stay on top of hotfixes. More will be discussed about this later on. End-of-life timelines should help customers in planning for these changes. Licensed versions of XenServer need to fall under active Customer Success Services (CSS) contracts to be eligible for access to the corresponding types of updates and releases that fall outside of current product cycles.


At a Glance

The summary of how various XS versions are treated looking forward is presented in the following table, which provides an overview at a glance.

XenServer Version

Active Hotfix Cut-Off

Action Required

6.2 & 6.5 SP1

EOL June 2018

None. Hotfixes will continue to be available as before until EOL.


December 2017

Upgrade or buy CSS.


December 2017

Upgrade or buy CSS (to stay on the LTSR stream by deploying CU1)


At release of next CR

Upgrade to latest CR, or buy CSS to obtain 4 months more of hotfixes (and thus skip one CR)


In the future, the only way to get hotfixes for XS 7.2 without paying for a CSS will be to upgrade to the most current CR once released. Customers who do pay for CSS will also be able to access hotfixes on 7.2 for a further four months after the next CR is released. This would in principle allow you to skip a release and transition to the next one beyond it.


What Stays the Same

While changes are fairly significant for especially users of the free edition of XS, some aspects will not change. These include:

·        There will still be free versions of XenSever released as part of the quarterly CR cycle.

·        The free version of XS can still be updated with all released hotfixes until the next CR becomes generally available.

·        XS code still remains open source.

·        Any hotfixes that have already been released will remain publically available.

·        There will be no change to XS 6.2 through 6.5 SP1. They will only receive security hotfixes at this point, anyway, until being EOLed in June 2018.


In Summary 

First off, stay calm and keep on maintaining XS pretty much as before except for the cut-offs on how long hotfixes will be available for non-paid-for versions. The large base of free XS users should not fear that all access to hotfixes is getting completely cut off. You will still be able to maintain your installation exactly as before, up to the point of a new CR. This will take place roughly every quarter. This is a pretty small amount of extra effort to keep up, especially given how much easier updating has become, in particular for pools. Most users would do well to keep up with current releases to benefit from improvements and to not have to apply huge numbers of hotfixes whenever installing a base version; the CR releases will include all hotfixes accumulated up to that point, which will be more convenient.


What To Do 

Any customers who already have active CSS contracts will be unaffected by any of these changes. Included are servers that support actively licensed XenApp/XenDesktop instances, who by the way, get all the XS Enterprise features incorporated into the latest XS releases – for all editions of XA/XD. Customers on XS 7.0 and 7.1 LTSR who have allowed their software maintenance to lapse and have not renewed/upgraded to CCS subscriptions should seriously consider that route.

Citrix will provide additional information as these changes are put into place.

Continue reading
5675 Hits

From the Field: XenServer 7.X Pool Upgrades, Part 2: Updates and Aftermath

The XenServer upgrade process with XenServer 7 can be a bit more involved than with past upgrades, particularly if upgrading from a version prior to 7.0. In this series I’ll discuss various approaches to upgrading XS pools to 7.X and some of the issues you might encounter during the process based on my experiences in the field.

Part 1 dealt with some of the preliminary actions needed to be taken into consideration and the planning process, as well as the case for a clean installation. You can go back and review it here. Part 2 deals with the alternative, namely a true upgrade procedure, and what issues may arise during and after the procedure.

In-Place Upgrades to XS 7

An in-place upgrade to XS 7.X is a whole different beast compared to where the OS is overwritten from scratch. Mostly, it depends on whether or not you wish to retain the original partition scheme or go with the newer, larger, and more flexible disk partition layout. The bottom line is despite the added work and potential issues, you may as well go through with the update as it will make life easier later down the road.

Your choices here are to retain the current disk partitioning layout or to switch to the newer partition layout. If you want to know why the new XS disk layout changed, review the “Why was the XenServer 7 Disk Layout Changed?” section in Part 1 of this blog series.

In my experience, issues found with in-place upgrades cover five areas – I’m going to cover them as follows: 

  1. Pre-XS 7 Upgrade, Staying the Course with the Original Partition Layout
  2. The 72% Solution
  3. The (In)famous Dell Utility Partition
  4. XS 7.X to XS 7.X Maintaining the Current Partition Layout
  5. XS 6.X or7.X to XS 7.X With a Change to the New Partition Layout (plus possible issues on Dell Servers with iDRAC modules)

Pre-XS 7 Upgrade, Staying the Course with the Original Partition Layout

If you choose to stick with the original partition layout, the rest of your installation experience – whether you go with the rolling pool upgrade, or conventional upgrade – will be pretty much the same as before.

As with any such upgrade, make sure to check ahead of time that the pool is ready for this undertaking: 

  • XenCenter has been upgraded 
  • Assure the proper backups of VMs, metadata, etc. have been performed 
  • That all VMs are agile or in the case of a rolling pool upgrade, at least shut down on all local SRs. 
  • All hosts must not only be running the same version of XenServer, but must have the identical list of hotfixes applied. 

Depending on the various ages of the hosts and how hotfixes were applied, you could run into the situation where there is a discrepancy. This can be particularly frustrating if an older hotfix has since been superseded and the older version is no longer available, yet shows up in the applied hotfix list. Though I’d only recommend this in dire circumstances (such as this, where the XenServer version is going to be replaced by a new one anyway) there are ways to make XenServers believe they have had the same specific patches applied by manipulating the contents of /var/update/applied directory (see, for example, this forum thread).

One thing you might run into is the following issue, covered in the next section, which incidentally appears to be independent of whichever partition layout you use.

The 72% Solution

This is not really so much of a solution, but rather a warning as to what you may encounter and what to do before you potentially run into this or what to do afterwards should you encounter it. 

This issue apparently crops up frequently during the latter part of the installation process and is characterized by a long wait with a screen showing 72% completion of the installation that can last anywhere from about five minutes to over an hour in extreme cases. 

One apparent cause of this is if you have an exceedingly large number of message files on your current version of XS. If you have the chance, it would be worthwhile taking the time to examine the contents of the area where these are stored and manually clean up any ancient versions within those areas under /var/lib/xcp/blobs/ in particular as most of these reside under the “messages” and “refs” subdirectories, paying attention also to the symbolic links.

If you do get stuck in the midst of a long-lasting installation you can escape out by pressing ALT+F2 to get into the shell and check the installer logs under /tmp/install-log to see if it has thrown any errors (thanks, Nekta -- @nektasingh -- for that tip!). If all looks OK, continue to wait until the process completes.

The (In)famous Dell Utility Partition

Using the rolling pool upgrade for the first time, I ran into a terrible situation in which the upgrade proceeded all the way to the very end and just as the final reboot was about to happen, I got this error popping up on the console:

Which read:

An unrecoverable error has occurred. The error was:

Failed to install bootloader: installing for i386-pc platform.

/usr/sbin/grub-install: warning: your embedding area is unusually small. core.img won’t fit in it..

/usr/sbin/grub-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However blocklists are UNRELIABLE and their use is discouraged..

/usr/sbin/grub-install: error: will not process with blocklists.

You can imagine the reaction.

What caused this and what can you do about it? After all, this was something I’d never encountered before in any previous XS upgrades.

It turns out that the reason for this is the Dell utility partition (see, for example, this link). The Dell utility partition is a five-block partition that Dell puts on for its own purposes at the beginning of many of its servers as they ship. This did not interfere with any installs up to and including XS 6.5 SP1, hence this came to me as a total surprise when I first encountered in during a XS 7 upgrade. 

And while this wasn't an issue initially or in one particular upgrade which for whatever reason managed to squeak by without any errors being reported whatsoever, it's too small to hold the initial configuration needed to do the installation under most circumstances when installing XS 7.

What's bad is that the XenServer installation doesn't apparently perform any pre-checks to see if that first partition on the disk is big enough. This is the case for both UEFI and Legacy/BIOS boots.

The solution was simply to delete that sda1 partition altogether using fdisk and re-install. Deleting the partition can be performed live on the host prior to the installation process.

You can then successfully bypass this issue. I have performed this surgical procedure on a number of hosts with success and have not experienced any adverse effects; you do not even require a vorpal sword to accomplish this task.

Possibly future upgrade processes will be better accommodating and should perform a pre-check for this along with looking at other potential inconsistencies such as the lack of VM agility or uniformity of applied hotfixes.

XS 6.X or 7.X to XS 7.X With a Change to the New Partition Layout 

This is the trickiest area where most, including me, seem to encounter issues.

One point that should be clarified right away is: 

Any local partition on or partly contained on the system disk to be upgraded is going to get destroyed in the process. 

There are no two ways about this. Hence, plan ahead: either 

  • Storage Xenmotion any local VMs to pooled storage, or
  • Export them. 

If you want or need to preserve your existing local storage SRs, you’ll have to stay with the original partition scheme.

Should you decide to update the new partition scheme, before the upgrade, you will need to perform the following action:

# touch /var/preserve/safe2upgrade
The upgrade process (to preserve the pool) will need to be performed using the rolling pool upgrade. The caveat is that if anything goes wrong during any part of the upgrade process, you will have to exit and try to start over. The end point can vary from it being simple to reconvene from where you left off, to having things in a broken state, depending on the circumstances.The pre-checks performed are supposed to catch issues beforehand and allow you to take care of them before launching into the upgrade process, but these do not always trap everything! Above all, make sure that:
  • All VMs are agile, i.e. that there are no VMs running or even resident on any local storage
  • High Availability (HA) and Workload Balancing (WLB) are disabled
  • You have created metadata backups to at least one pooled storage device or exported it to an external location 
  • You have plenty of space to hold VMs on whatever hosts remain in your pool, figuring you’ll have one out at any given point and that initially, only the master will be potentially able to take on VMs from whichever hosts is in the process of being upgraded 
Preferably also have recent backups of all your VMs. It’s generally also a good idea to keep good notes about your various host configurations, including all physical and virtual networks, VLANs, iSCSI and NFS connection and the like.  I’d recommend doing an “export resource data” for the pool, which has been available as a utility since XS 6.5 and can run from XenCenter.To export resource data:
  1. In the XenCenter Navigation pane, click Infrastructure and then click on the pool. 
  2. From the XenCenter menu, click Pool and then select Export Resource Data. 
  3. Browse to a location where you would like to save report and then click Save.
You can chose between an XLS or CVS output. Note that this feature is only available on XenCenter with paid-for licensed version of XenServer. However, it can also be run via the CLI for any (including free) version of XS 6.5 or newer using:
# xe pool-dump-database file-name=target-output-file-name
Being prepared now for the upgrade, you may still run into issues along the way, in particular if you run out of space to move VMs to a different server or if the upgrade process hangs. If the master server manages to make it through the upgrade and the upgrade process fails at some point later, one of the first consequences will be that you will no longer be able to add any external hosts to the pool because the pool will be in a mixed state of hosts running different XS versions. This is not a good situation and makes it very difficult under some circumstances to recover from.

Recommended Process for Changing XenServer Partition Layout

Through various discussions on the XenServer discussion forum as well as some of my own experimentation, this is what I recommend as the most solid and reliable way of doing an upgrade to XS 7.X with the new partition layout coming from a version that still has the old layout. It will take quite a bit longer, but is more reliable.  
In addition to all the preparatory steps listed above:

In addition to all the preparatory steps listed above:

  • You will be needing to eject all hosts from the pool at one point or another, so make sure you have carefully recorded the various network and other settings, in particular some which are not retained as such in the metadata exports, such as the individual iSCSI network connections or NFS connections. 
  • Copy your /etc/fstab file and also be sure to check for any other customizations you may have done, such as cron jobs or additions to /etc/rc.local (and also note that rc.local does not run on its own under XS 7.X, so you will need to manually enable it to do so – see the section “Enabling rc.local to Run On Its Own” below).

Once your enhanced preparation is complete: 

  1. Start with your pool master and do a rolling pool upgrade to it. Do not attempt to upgrade to the new partition layout! After it reboots, it should still have retained the role of pool master. Note that the rolling pool upgrade will not allow you to pick which host it will upgrade next beyond the master so be certain that any of these hosts can have all its VMs fit on the pool master. If necessary move some or all the VMs on the pool master onto other hosts within the pool before you commence the upgrade procedure.
  2. Pick another host and migrate all its VMs to the pool master.
  3. Follow this procedure for all hosts until the pool is completely upgraded to the identical version of XS 7.X on all pool members. You can either continue to use the rolling pool upgrade or if desired, switch to manual upgrade mode.
  4. Making sure you’ve carefully recorded all important storage and network information on that host, migrate all VMs off a non-master host and eject it from the pool.
  5. Touch the file /var/preserve/safe2upgrade and plan on the local SR being wiped and re-created. Then shutdown the host and perform a standalone installation to that just ejected host. It will have the new partition layout.
  6. Reduce the network settings on this host to just a single primary management interface NIC, one that matches the same one it had initially. Rejoin this just upgraded host back into the pool. Many of the pool metadata settings should automatically get recreated, in particular any of the pooled resources. Update any host-specific network settings as well as other customizations.
  7. Continue this process for the remainder of the non-master hosts.
  8. When these have all been completed, designate a new pool master using the command “xe pool-designate-new-master host-uuid=new-master-uuid”. Make sure the transition to the new pool master is properly completed. 
  9. Eject what was the original pool master and perform the standalone upgrade and rejoin the host to the pool as before.
  10. Celebrate your hard work!
While this process will take easily two times as long as a standard upgrade, at least you know that things should not break so badly that you may have to spend hours of additional time making things right. As a side benefit (if you want to consider it as such), it will also force you to take stock of how your pool is configured and require you to take careful inventory, In the event of a disaster, you will be grateful to have gone through this process as it may be very close to what you may have to go through under less controlled circumstances!I have done this myself several of times and had it work correctly each time.

An Additional Item: If the Console Shows No Network Configured on Dell Servers with iDRAC modules

This condition can show up unexpectedly and while normally something like this can be handled if the host is found to be in emergency mode with an “xe pool-recover-slaves” command, that’s not always the case. And even more oddly, if instead you ssh in and run xsconsole from the CLI, all looks perfectly normal, including all the network settings that appear present and correct and also match the settings visible in XenCenter. This condition, as far as I know, seems unique to Dell servers and was seen with iDRAC 7, 8 and 9 modules. Here's what it looks like:

The issue here appears to have been a change in behavior that kicked in starting with XenServer 7.1 and hence may not even be evident in XS 7.0.

The fix turned out to be an upgrade all the BIOS/firmware and iDRAC configurations. In this particular case, I made use of the method I described in this blog entry and that took care of it.  Note that this still does not seem to consistently address this issue, in particular with some older hardware (e.g., R715 with an iDRAC 6, even after updating to DSU 1.4.2, BIOS 3.2.1, but being stuck at iDRAC version 1.97 – apparently not upgradeable).

Enabling rc.local To Run On Its Own

The rc.local file is not automatically executed by default under RHEL/CentOS 7 installations, and since XS 7 is based on CentOS 7, it is no exception. If you wish to customize your environment and assure this file is executed at boot time, you will have to manually enable /etc/rc.local to run automatically on reboot on XenServer 7.X and to do so, will need to run these two commands:

# chmod u+x /etc/rc.d/rc.local
# systemctl start rc-local

You can verify that rc.local is now running with the following command, which in turn should produce output similar to what is shown below:

# systemctl status rc-local

   rc-local.service - /etc/rc.d/rc.local Compatibility
 Loaded: loaded (/usr/lib/systemd/system/rc-local.service; static; vendor preset: disabled)
Active: active (exited) since Sun 2017-06-18 09:09:50 MST; 1 weeks 6 days ago
Process: 4649 ExecStart=/etc/rc.d/rc.local start (code=exited, status=0/SUCCESS)

A Final Word

Once again, I will reiterate that feedback is always appreciated and the XenServer forum is one good option. Errors, on the other hand, should be reported to the XenServer bugs site with as much information as possible to help the engineers understand and reproduce the issues.

I sincerely hope some of the information in this series will have been useful to some degree.

I would like to most gratefully acknowledge Andrew Wood, fellow Citrix Technology Professional, for the review of and constructive additions to this blog.


Recent comment in this post
Tobias Kreidl
I would like to add that an option for a more efficient, yet safe update procedure, would be to follow what's in the article, but ... Read More
Thursday, 06 July 2017 19:00
Continue reading
12006 Hits
1 Comment

From the Field: XenServer 7.X Pool Upgrades, Part 1: Background Information and Clean Install

The XenServer upgrade process with XenServer 7 can be a bit more involved than with past upgrades, particularly if upgrading from a version prior to 7.0. In this series I’ll discuss various approaches to upgrading XenServer (XS) pools to 7.X and some of the issues you might encounter during the process based on my experiences in the field.

Part 1 deals with some of the preliminary actions that need to be taken into consideration and the planning process, as well as the case for a clean installation. Part 2 deals with the alternative, namely a true in-place upgrade procedure, and what issues may arise during and after the procedure.


Why Upgrade to XenServer 7?

XenServer (XS) 7 has been out for a little over a year now and is currently up to release 7.2. A number of improvements in scaling, functionality and features are available in XS 7, which should encourage users to upgrade to take advantage of them.


First Step in Any XS Upgrade

The first step in any XenServer upgrade is to upgrade your XenCenter to at least the minimum version required to support the newest installation of XenServer you are or will be accessing.

This step is crucial. Failure to do this causes puppies to die. This is because older versions of XenCenter won’t connect to newer versions of XS and in many cases, new features are built into the newer versions of XenCenter that are not available in older versions and support important components in the newer XS releases, including hotfix and rolling pool upgrade options, changes in licensing, and others such as supported pooled storage options. In short, your upgrade process will rapidly come to a screeching halt and there may be potential damage done unless you are working with the proper version of XenCenter.

It is also highly recommended to make backups of anything and everything prior to upgrading. More on that will be discussed later.


Summary of Pre-Upgrade Checks

From experience, the following are key activities to perform before upgrading any XenServer pool:

XenCenter has been upgraded to at least the version that does or will match your newest pool

  • Assure the proper backups of virtual machines (VMs), metadata, etc. have been performed
  • Make sure that all VMs are agile (can be migrated to other hosts within the pool) or in the case of a rolling pool upgrade, at least shut down on all local SRs
  • All hosts must not only be running the same version of XenServer, but must have the identical list of hotfixes applied
  • Clean up old messages in /var/lib/xcp/blobs/

I’d also recommend doing an “export resource data” for the pool, which has been available as a utility since XS 6.5 and can run from XenCenter.

To export resource data:

  1. In the XenCenter Navigation pane, click Infrastructure and then click on the pool.
  2. From the XenCenter menu, click Pool and then select Export Resource Data.
  3. Browse to a location where you would like to save the report and then click Save.

You can chose between an XLS or CVS output. Note that this feature is only available on XenCenter with paid-for licensed versions of XenServer. However, it can also be run via the CLI for any (including free) version of XS 6.5 or newer using:

# xe pool-dump-database file-name=target-file-name


Benefits of a XenServer 7 Upgrade

XenServer 7 is a significant jump from XS 6.5 SP1 in many ways, including improvements in scale and performance. It also has added a number of very nice new features (see what's new in XS 7; what's new in XS 7.1; XS 7.2 ).

While any upgrade can be a bit intimidating, ones that jump to a whole new edition can be more complicated and given the changes involved here, this case was no exception. I would also recommend trying to verify your hardware is present on the HCL though it’s often slow to get populated. If in doubt and you have the extra resources, take as close as possible a spare server and try to do an installation on it first to make sure it’s compatible. If you don’t have such a machine and have the spare capacity, eject one of the hosts from your pool and use it to test out the upgrade. Before performing an upgrade on a production server or pool, it is always recommended to try this out first on a test configuration.

I do not recommend at this time the rolling pool or standalone upgrade that involves switching at the same time from the old to the new partition layout. Overall experiences within the user community have not been consistently positive.

It is hoped that the preparatory steps that are recommended before undertaking an upgrade do not prove to be too daunting. This series is intended to help with those preparations as well as give guidelines and tips how to go about the process and what to do if issues arise. There are clearly a number of ways to tackle upgrades and there are undoubtedly numerous other possible approaches. One might, for example, storage XenMotion VMs to other pools or make use of temporary pooled storage as part of the process. The detaching and attaching pooled storage to servers already running a newer version of XenServer is another possible route to explore.

Experimentation is always something that should be encouraged, albeit not on production systems!

Feedback is always appreciated and the XenServer forum is one good option. Errors, on the other hand, should be reported to the XenServer bugs site with preferably sufficient detail to allow the engineers to be able to understand and reproduce the issues.


XS 7.X Leveraging a Clean Installation as an Upgrade Component

A clean install can become part of an upgrade procedure if for example an upgraded pool already exists and individual hosts or hosts from a whole different pool are to be merged into a pool running a newer version of XenServer. Hence talking about this option is relevant to upgrading. You can potentially storage Xenmotion VMs over to a new pool, then eject and dissolve individual pool members, perform a clean installation on them, and join the hosts to the new pool. Of course the target pool must have adequate resources to hold and run the VMs being moved over until the new hosts can be configured and added. This process might involve leveraging temporary external storage, for example, by creating an NFS SR.

The easiest installation scenario is a clean install, which has thankfully always been the case and this approach may work well in some instances. This is obviously suited best for a standalone server or as mentioned above, creating a server that will be joined to an existing pool.

Regardless of which upgrade process you undertake, be sure to review your VMs afterwards to see if XenTools need to be updated.


A Few Words About the XenServer 7 New Disk Layout

The main item to be aware of is that a clean installation will always create the new GPT partition layout as follows:



Size Purpose
4GB log partition (syslogd and other logs)
18GB backup partition for dom0 (for upgrades)
18GB XenServer host control domain (dom0) partition
0.5GB UEFI boot partition
1GB swap partition


This adds up to 41.5 GB. There can be a sixth partition which is optionally used for creating a local storage repository (SR) or a portion of a local SR if spanning it to other storage devices. See also James Bulpin's XS 7 blog for more details about the XS 7 architecture.

Here is what a real-life partition table looks like after performing an installation. Note that the space for a local SR ends up in partition 3 (in this case, a default LVM partition 94.6 GB in size):

# fdisk -l /dev/sda

WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sda: 146.2 GB, 146163105792 bytes, 285474816 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: gpt

#         Start            End          Size  Type       Name

 1     46139392     83888127     18G  Microsoft basic

 2       8390656     46139391     18G  Microsoft basic

 3     87033856   285474782   94.6G  Linux LVM

 4     83888128     84936703   512M  BIOS boot parti

 5            2048       8390655       4G  Microsoft basic

 6     84936704     87033855       1G  Linux swap


Why was the XenServer 7 Disk Layout Changed?

The XS 6.5 SP1 and earlier partition scheme was much smaller and simpler:

4 GB XenServer host control domain (dom0) partition

4 GB backup partition for dom0 (for upgrades)

The rest of the space was again available for use as a local SR in a third partition.

The default partition layout was changed under XS 7 for a number of reasons. For one, most storage devices these days start with a capacity of few hundred GB and so it makes sense to be able to allocate more space for the XenServer installation itself beyond just 8 GB. In that regard, running out of space has been a common problem caused by either a lot of hotfixes being stored or the accumulation of a lot of log files. Isolating /var/log to its own partition now helps keep XS from crashing in case it fills up. Before, /var/log was just another subdirectory on the system “/” partition and inevitably resulted in a system crash when the partition became 100% full. A separate UEFI boot area was also added. Plus, the Linux swap space was assigned to its own partition. In short, there was no reason not to make use of more space and provide an improved partitioning scheme that would also scale better for larger pools containing a lot of VMs.

Note: One idiosyncrasy of XS 6.5 SP1 is that it switched to a GUID partition table in order to use more than 2 TB of local storage if available, but this has evidently been undone in 7.0 since GPT is no longer limited as such when running a 64-bit OS based on CentOS 7.

Note: There are changes in the minimum host physical system requirements, including a recommended minimum of 46 GB of disk space. This will impact environments that used smaller partitions or SSDs for the XS OS.


Stage Complete: Level Up!

Well done – you’ve completed Part 1. In Part 2, I’ll deal with the alternative to leveraging a clean install -- a true in-place upgrade procedure, and what issues may arise during and after the procedure.

Once again, I will reiterate that feedback is always appreciated and the XenServer forum is one good option. Errors, on the other hand, should be reported to the XenServer bugs site with as much information as possible to help the engineers understand and reproduce them.


Continue reading
10303 Hits

XenServer 7.2 now available!

Hello XenServer community!

The XenServer team is proud to announce the release of version 7.2, which includes an array of improvements that further simplify and refine the user experience and enable even greater platform management scalability.

Click here to learn more about the new features available in XenServer 7.2.

See you on the XenServer page on!

Andy M.


Recent Comments
Awesome! All the download pages link to 7.1, tho.
Thursday, 25 May 2017 23:23
Andy Melmed
To learn about all the really cool things XenServer 7.2 has to offer our customers, visit Read More
Wednesday, 31 May 2017 21:18
Andy Melmed
Hi Christian, The links on the downloads page to access v7.2 have been updated. Thank you, Andy... Read More
Monday, 05 June 2017 13:31
Continue reading
18367 Hits

Creating backups with XenServer

Backup is an essential part of the business workflow for many of our customers - be it SMB, Enterprise Server Virtualisation or Virtual Desktop Infrastructure. Making the backup experience smoother is high up on our wishlist at XenServer Engineering and the delivery of improved VM import/export performance in XS 7.1 shows our commitment to that end. To continue improving our services supporting the backup ecosystem, we would like to better understand how you use backup with XenServer


  • How often do you backup? Do you have multiple jobs for monthly, weekly, daily backups?

  • How do you create your backups?

    • Use VM Export to backup VM metadata + disks

    • Snapshot at the VM level and use transfer/service VM to read off the snapshots

    • Use vdi-export to create differential disks (.vhd)

  • Do you use a third-party vendor for handling your backups?

  • Would support for incremental backups be useful for your use case?

Please leave a comment with your answers and any issues you may have with your backup experience today. We look forward to hearing from you!

Thank you,



Recent Comments
I use storage replication for sr and bacula for vm data. I still afraid of coalesce issue for using snapshot export.
Friday, 07 April 2017 02:30
Chandrika Srinivasan
Hi Beck, Is there a specific issue you are facing with coalesce? Which version of XenServer are you using? -Chandrika... Read More
Tuesday, 02 May 2017 14:02
Olivier Lambert
Check this: Should be integrated soon in our backup... Read More
Friday, 12 May 2017 19:23
Continue reading
27867 Hits

XenCenter 7.1 update now available!

A hotfix (XS71E001) has been released for customers using XenCenter as the management console for their XenServer 7.1 virtual environments. 

This hotfix offers improvements in XenCenter UI responsiveness, as well as several fixes associated with host health check analysis, status reports and updates. Additional information pertaining to this hotfix can be found here.

As always, we encourage customers read the hotfix release notes and install the hotfix to avoid any of the issues described in the notes.


Recent comment in this post
Tobias Kreidl
Note that this distribution includes both the XenCenter binary as well as a hotfix that needs to be applied to the XenServer host,... Read More
Friday, 07 April 2017 12:53
Continue reading
8593 Hits
1 Comment

Introducing... XenServer 7.1!

We are pleased to announce the release of XenServer 7.1!

Click here to learn about the new features and enhancements available in 7.1.

As is customary with every new release, we encourage you to give v7.1 a spin and report any issues via

Note: We ask that you target this release exclusively for new defect reports[*].

Thank you and enjoy the latest release!

[*]In case of problems with earlier releases, pre-XS v7.0 and outside of paid support, then we recommend you upgrade to the XS v7.x series.  




Recent Comments
Andrew Halley
See here for which features are available in which versions : Read More
Friday, 24 February 2017 17:41
Hey, Great news! The Download links still reflect 7.0 release, tho. Any chance to get a download link? -Chris.... Read More
Friday, 24 February 2017 22:07
Andrew Halley
We're working on it - now done!
Monday, 27 February 2017 16:51
Continue reading
9188 Hits

Staying Ahead of the Curve

Are you looking to improve the performance of your virtual servers and desktops?

Could your hypervisor use a boost when it comes to supporting graphics-intense applications?

Are you in need of an advanced security technology that offers a unique way of detecting and blocking sophisticated attacks against your data center before they cause any damage to your business?

Would you like to simplify the maintenance of your hosting infrastructure?

Does the idea of optimizing the performance, scalability, management and cost-savings of your application and desktop delivery solutions through the combination of an industry-leading hypervisor and industry-leading HCI platforms sound interesting to you?

Would you feel more comfortable knowing your hosting infrastructure was fully-supported for the next 10 years?

If you answered "yes" to any of the above, click here to learn more!

Until next time,



Continue reading
3358 Hits

XenServer High-Availability Alternative HA-Lizard

XenServer High-Availability Alternative HA-Lizard


XenServer (XS) contains a native high-availability (HA) option which allows quite a bit of flexibility in determining the state of a pool of hosts and under what circumstances Virtual Machines (VMs) are to be restarted on alternative hosts in the event of the loss of the ability of a host to be able to serve VMs. HA is a very useful feature that protects VMs from staying failed in the event of a server crash or other incident that makes VMs inaccessible. Allowing a XS pool to help itself maintain the functionality of VMs is an important feature and one that plays a large role in sustaining as much uptime as possible. Permitting the servers to automatically deal with fail-overs makes system administration easier and allows for more rapid reaction times to incidents, leading to increased up-time for servers and the applications they run.

XS allows for the designation of three different treatments of Virtual Machines: (1) always restart, (2) restart if possible, and (3) do not restart. The VMs designated with the highest restart priority will be the first to be attempted to restart and all will be handled, provided adequate resources (primarily, host memory) are available.  A specific start order, allowing for some VMs to be checked to be running before others, can also be established. VMs will be automatically distributed among whatever remaining XS hosts are considered active. Where necessary, note that hosts that contain expandable memory will be shrunk down to accommodate additional hosts and those hosts designated to be restarted will also be run with reduced memory, if necessary. If additional capacity exists to run more VMs, those designated as “start if possible” will be brought online. Whichever VMs that are not considered essential typically will be marked as “do not restart” and hence will be left “off” had they been running before, requiring any of those desired to be restarted to be done manually, resources permitting.

XS also allows for specifying the minimum number of active hosts to remain to accommodate failures; larger pools that are not overly populated with VMs can readily accommodate even two or more host failures.

The election of what hosts are “live” and should be considered active members of the pool follows a rather involved process of a combination of network accessibility plus access to an independent designated pooled Storage Repository (SR) that serves as an additional metric. The pooled SR can also be a fiber channel device, being independent of Ethernet connections. A quorum-based algorithm is applied to establish which servers are up and active as members of the pool and which -- in the event of a pool master failure -- should be elected the new pool master.



Without going into more detail, suffice it to say that this methodology works very well, however requiring a few prerequisite conditions that need to be taken into consideration. First of all, the mandate that a pooled storage device be available clearly means that a pool consisting of hosts that only make use of local storage will be precluded. Second, there is also a constraint that for a quorum to be possible, it is required to have a minimum of three hosts in the pool or HA results will be unpredictable as the election of a pool master can become ambiguous. This comes about because of the so-called “split brain” issue ( which is endemic in many different operating system environments that employ a quorum as means of making such a decision. Furthermore, while fencing (the process of isolating the host; see for example is the typical recourse, the lack of intercommunication can result in a wrong decision being made and hence loss of access to VMs. Having experimented with two-host pools and the native XenServer HA, I would say that an estimate of it working about half the time is about right and from a statistical viewpoint, pretty much what you would expect.

This limitation is, however, still of immediate concern to those with either no pooled storage and/or only two hosts in a pool. With a little bit of extra network connectivity, a relatively simple and inexpensive solution to the external SR can be provided by making a very small NFS-based SR available. The second condition, however, is not readily rectified without the expense of at least one additional host and all the connectivity associated with it. In some cases, this may simply not be an affordable option.



For a number of years now, an alternative method of providing HA has been available through the program package provided by HA-Lizard ( , a community project that provides a free alternative that is neither dependent on external SRs nor requires a minimum of three hosts within a pool. In this blog, the focus will be on the standard HA-Lizard version and because of the particularly harder-to-handle situation of a two-node pool, it will also be the subject of discussion.

I had been experimenting for some time with HA-Lizard and found in particular that I was able to create failure scenarios that needed some improvement. HA-Lizard’s Salvatore Costantino was more than willing to lend an ear to the cases I had found and this led further to a very productive collaboration on investigating and implementing means to deal with a number of specific cases involving two-host pools. The result of these several months of efforts is a new HA-Lizard release that manages to address a number of additional scenarios above and beyond its earlier capabilities.

It is worthwhile mentioning that there are two ways of deploying HA-Lizard:

1) Most use cases combine HA-Lizard and iSCSI-HA which creates a two-node pool using local storage while maintaining full VM agility with VMs being able to run on either host. In this case, DRBD ( is implemented in this type of deployment and it works very well making use of the real-time storage replication.

2) HA-Lizard, only, is used with an external Storage Repository (as in this particular case).

Before going into details of the investigation, a few words should go towards a brief explanation of how this works. Note that there is only Internet connectivity (the use of a heuristic network node) and no external SR, so how is a split brain situation then avoidable?

This is how I'd describe the course of action in this two-node situation:

If a node sees the gateway, assume it's alive. If it cannot, assume it's a good candidate for fencing. If the node that cannot see the gateway is the master, it should internally kill any running VMs and surrender its ability to be the master and fence itself. The slave node should promote itself to master and attempt to restart any missing VMs. Any that are on the previous master will probably fail though, because there is no communication to the old master. If the old VMs cannot be restarted, eventually the new master will be able to restart them regardless after a toolstack restart. If the slave node fails by not being able to communicate with the network, as long as the master still sees the network and not the slave’s network, it can assume the slave needs to fence itself, kill off its VMs and assume that they will be restarted on the current master. The slave needs to realize it cannot communicate out, and therefore should kill off any of its VMs and fence itself.

Naturally, the trickier part comes with the timing of the various actions, since each node has to blindly assume the other is going to conduct a sequence of events. The key here is that these are all agreed on ahead of time and as long as each follows its own specific instructions, it should not matter that each of the two nodes cannot see the other node. In essence, the lack of communication in this case allows for creating a very specific course of action! If both nodes fail, obviously the case is hopeless, but that would be true of any HA configuration in which no node is left standing.

Various test plans were worked out for various cases and the table below elucidates the different test scenarios, what was expected and what was actually observed. It is very encouraging that the vast majority of these cases can now be properly handled.


Particularly tricky here was the case of rebooting the master server from the shell, without first disabling HA-Lizard (something one could readily forget to do). Since the fail-over process takes a while, a large number of VMs cannot be handled before the communication breakdown takes place, hence one is left with a bit of a mess to clean up in the end. Nevertheless, it’s still good to know what happens if something takes place that rightfully shouldn’t!

The other cases, whether intentional or not, are handled predictably and reliably, which is of course the intent. Typically, a two-node pool isn’t going to have a lot of complex VM dependencies, so the lack of a start order of VMs should not be perceived as a big shortcoming. Support for this feature may even be added in a future release.



HA-Lizard is a viable alternative to the native Citrix HA configuration. It’s straightforward to set up and can handle standard failover cases with a selective “restart/do not restart” setting for each VM or can be globally configured. There are a quite a number of configuration parameters which the reader is encouraged to research in the extensive HA-Lizard documentation. There is also an on-line forum which serves as a source for information and prompt assistance with issues. This most recent release 2.1.3 is supported on both XenServer 6.5 and 7.0.

Above all, HA-Lizard shines when it comes to handling a non-pooled storage environment and in particular, all configurations of the dreaded two-node pool configuration. From my direct experience, HA-Lizard now handles the vast majority of issues involved in a two-node pool and can do so more reliably than the non-supported two-node pool using Citrix’ own HA application. It has been possible to conduct a lot of tests with various cases and importantly, and to do so multiple times to ensure the actions are predictable and repeatable.

I would encourage taking a look at HA-Lizard and giving it a good test run. The software is free (contributions are accepted) and it is in extensive use and has a proven track record.  For a two-host pool, I can frankly not think of a better alternative, especially with these latest improvements and enhancements.

I would also like to thank Salvatore Costantino for the opportunity to participate in this investigation and am very pleased to see the fruits of this collaboration. It has been one way of contributing to the Citrix XenServer user community that many can immediately benefit from.







Recent comment in this post
JK Benedict
I hath no idea why more have not read this intense article! As always: bravo, sir! BRAVO!
Wednesday, 04 January 2017 12:43
Continue reading
14570 Hits
1 Comment

About XenServer

XenServer is the leading open source virtualization platform, powered by the Xen Project hypervisor and the XAPI toolstack. It is used in the world's largest clouds and enterprises.
Technical support for XenServer is available from Citrix.