XenServer Virtual Machines (VMs) certainly need no introduction, but even if you do not pardon the pun above, they still contain a lot of specialized and individualized information about their sizes, network connections, and myriad other settings that are generally not readily exposed, yet are integral to the operation and functionality of each VM. This blog entry is not intended to take a deep dive into the several hundred parameters that are defined for VMs, but rather just talk a bit about how to save, extract and potentially restore VM information based on them.
VM Metadata Backups
The purpose of backing up the metadata from a VM is to help you understand how it’s configured without having to search through the list of parameters accessible via various “xe” commands or XenAPI calls that require some programming efforts, plus also allow you to potentially track changes to your VMs without necessitating a full VM export/backup each time. You may not want or need to restore a VM from a full backup, but rather just revert a few parameters back to older values. You might also want to monitor what sorts of changes have taken place over time and equate those to performance or other metrics. In short, a number of reasons to maintain relatively frequent metadata backups of VMs can be justified.
Getting VM Metadata
There are a number of ways to obtain VM metadata settings. One of these is with the standard “xe” command to extract parameters, either individually, in a comma-separated string of multiple queries, or all of them such as with:
# xe vm-list uuid|name-label=UUID|NAME-LABEL params=all
Here, either the UUID or name-label can be used to select the VM.
Some parameters can also not only be obtained via “xe vm-param-get” but also changed, using the complementary “xe vm-param-set” operator. This gives you access to modifying around 30 parameters and reading over 80 of them.
The XenCenter console, xsconsole, provides a direct way to back up and restore VM metadata. From the "Backup, Restore and Update" menu, you can next navigate to the "Backup Virtual Machine Metadata" option and choose from available SRs onto which you wish to create a metadata backup of all available VMs. Note, though, that it will only evidently create metadata backups of running VMs! Likewise, the restore operation can only be performed on various subsets of VMs and not on an individual VM. Conversely, the metadata restore operation apparently can be applied to both running and halted VMs, but again cannot be performed on an individual VM.
Another option is to make use of the XenAPI library and extract tokens using XenAPI calls and access them for example using constructs such as these:
vm_meta_status = gather_vm_meta(vm_object, full_backup_dir)
vm_record = session.xenapi.VM.get_record(vm_object)
vm_out = open ('%s/vm.cfg' % tmp_full_backup_dir, 'w')
vm_out.write('name_label=%s\n' % vm_record['name_label'])
vm_out.write('name_description=%s\n' % vm_record['name_description'])
vm_out.write('memory_dynamic_max=%s\n' % vm_record['memory_dynamic_max'])
vm_out.write('VCPUs_max=%s\n' % vm_record['VCPUs_max'])
vm_out.write('VCPUs_at_startup=%s\n' % vm_record['VCPUs_at_startup'])
This can be time-consuming if you want to keep identifying and modifying code to deal with any additions or changes, plus you may periodically have to update your API libraries.
Yet another option is to make use of the not-well-documented features within the “xe” command set associated with vm-export and vm-import utilities. It is possible to export just the metadata from a VM using the following syntax:
# xe vm-export metadata=true uuid=UUID-OF-VM) filename=/full_path/OUTPUT_FILE.XVA
This will create what is in essence a tar file containing a single file that captures more than 300 parameters! The XML code has to be extracted from this tarball, which contains a single file that is always named ova.xml and can be pulled from the XVA files with a basic tar command in which specifying the file to extract as ova.xml is optional, since it’s the one and only file within the tar file:
# tar –xf OUTPUT_FULE.XVA
tar: ova.xml: implausibly old time stamp 1969-12-31 17:00:00
Note that you may get this rather interesting message regarding the timestamp, which can be ignored. It may also turn out that the output file has absolutely no access permissions set, so you may want to run a “chmod 600 ova.xml” (or 644, etc.) to make it readable. You may also wish to rename it so it’s unambiguous and/or less likely to be overwritten.
For exported XVA files that are gzipped, you can extract the ova.xml file in a single operation with:
# tar xzf OUTPUT_FULE.XVA .gz
tar: ova.xml: implausibly old time stamp 1969-12-31 17:00:00
Once extracted, let’s take a look at the first part of the ova.xml file, which takes on a rather “ugly” appearance:
There is a wealth of information in here, but it’s not in a very friendly format. Fortunately, this can be readily rectified with the handy xmllint utility already present on XenServer (at least on 7.X):
# tar -xOf /ubuntu12-xs66-specialchars.XVA | xmllint –format - >/tmp/output_VM.xml
The ”-O” flag causes the output to be redirected to stdout and hence it can be piped to the xmllint utility, which in turn can generate a very nicely formatted and properly indented XML file. Note that the “-“ before the redirection “>” operator signifies the output of xmllint to go to stdout and if desired, the command can be abbreviated as such, in which case the output will just appear on the terminal. It will be several hundred lines long, so you may as well redirect the output into a file where you can more conveniently deal with reviewing that amount of information.
Here is what the first part of the formatted file really looks like:
OK, Great -- Now What?
Given the ability to now parse and peruse the XML metadata file associated with a particular VM, one could contemplate creating periodic backups of the VM metadata to have on hand in case one needs to reconstruct something or check if anything had changed. That’s all fine and good, but other than using “xe” commands or other means to change individual parameters, how does having these data help in the event of wanting to reconstruct or restore a VM?
The bottom line is that this feature has limited direct applications, though it does have a few. Consider the case of trying to use an XVA file that only contains the VM’s metadata to restore a VM. Note that the original VM must of course still exist or there will be nothing present to associate the VM storage with if a version of this VM cannot be found. However, if it is present, consider the following results:
# xe vm-import preserve=true filename=/ubuntu12-xs66-specialchars.XVA
The VM cannot be imported unforced because it is either the same version or an older version of an existing VM.
vm: 9538882a-c7e4-b8e5-c1f9-0d136f4a81b1 (TST-ubuntu12-vmtst3-xs66)
Perhaps as anticipated, it will fail as the “preserve=true” flag will first check for a duplicate VM and upon discovering it, flags it as a command that would overwrite the existing VM. That’s a good thing. Leaving off that flag, we next try:
xe vm-import filename=/ubuntu12-xs66-specialchars.XVA metadata=true
This should yield success, but what kind of success? What happens is the VM created using the “metadata=true” flag produces a new VM copy with the same name, but a different UUID for the VM that is “Created by template provisioner.” What has happened is just that a fast clone has been created. You will see that if you delete such a VM created that way, it will not show any storage devices associated with it and in XenCenter, it will therefore not ask you if you want to delete the associated storage.
This is not entirely without use, however as you can still make use of this VM and perhaps even compare its characteristics to the original. Furthermore, you can export the VM, and import it as a new VM in which case it will gain the properties of a full clone. At that point, dependence on the original no longer exists.
This exercise might be useful in debugging or checking parameter-based performance or other issues between the original and subsequent metadata modifications. Such headers may also be useful just for tracking historical uses of VMs, checking to see what IP addresses may have been assigned, and numerous other things.
The Full XVA Export
The discussion up to this point should result in a mental lightbulb turning on and raising the question, well, if I restore a full export of a VM, isn’t all this information already in there? Since clearly it has to be for a vm-import to work properly, an examination of a full XVA export will indeed reveal that it consists of many fairly small files, numbering at times many thousands, but always starting with our old friend, ova.xml, as we see from this sample output that lists the contents instead of extracting it:
# tar -tvf /exports/test-export.xva |less
---------- 0/0 29935 1969-12-31 17:00 ova.xml
---------- 0/0 1048576 1969-12-31 17:00 Ref:367/00000000
---------- 0/0 40 1969-12-31 17:00 Ref:367/00000000.checksum
---------- 0/0 1048576 1969-12-31 17:00 Ref:367/00000001
---------- 0/0 40 1969-12-31 17:00 Ref:367/00000001.checksum
---------- 0/0 1048576 1969-12-31 17:00 Ref:367/00000002
---------- 0/0 40 1969-12-31 17:00 Ref:367/00000002.checksum
---------- 0/0 1048576 1969-12-31 17:00 Ref:367/00000003
---------- 0/0 40 1969-12-31 17:00 Ref:367/00000003.checksum
---------- 0/0 1048576 1969-12-31 17:00 Ref:367/00000004
---------- 0/0 40 1969-12-31 17:00 Ref:367/00000004.checksum
The nice aspect of having the XVA file self-contained with all the metadata, as well as the data contents of the VM, makes this standalone file easy to move around and utilize as a backup.
The other nice aspect is that you, in fact, can use it as its own metadata storage mechanism and extract only that part of it if desired without needing to create a separate metadata backup (unless, of course, you want to do that more often and independently of a a vm-export). To extract just the metadata from this file shown above, all you need to do is to specify the embedded tar file name:
tar -xvf /exports/test-export.xva ova.xml
We now have the same metadata file content we had when running the vm-export command combined with the “metadata=true” option.
First off, it cannot be overstated that your XenServer environment should be backed up frequently and fastidiously, including both the pool metadata as well as the individual metadata for VMs. Even if you already have full exports of your VMs, having additional metadata can be useful for auditing purposes, as well as making it possible to check on parameters that are hard or impossible to glean through other means.
Nobody that I know was ever accused of creating too many backups.