Container Monitoring in Linux VMs
How to get started with Container Monitoring on CoreOS
1) Install and configure CoreOS
Provision the VM using the XenCenter "New VM" wizard. Select CoreOS as the operating system. Walk through the wizard. In the new Cloud Config Parameters tab, you will have the option to customize ssh_authorized_keys. There is a sub-setting "- ssh-rsa", that is commented out by default, which can be used to register personal public SSH key to be able to login into the VM with the "core" user later:
# - ssh-rsa
While adding your personal public key, you may also have noticed a separate "- ssh-rsa %XSCONTAINERRSAPUB%" setting in which the XenServer container management registers and SSH key for its’ management operations. This is mandatory for the container management and should not be modified.
After following the remainder of "New VM" wizard, you can use your preferred method to install CoreOS. One way is to download the CoreOs-iso from here to your ISOSR and boot the VM using that ISO. Subsequently run the following command on the VM-console, to install to the first harddisk:
sudo coreos-install –d /dev/xvda –o xen –C stable
Next you can reboot into the installed VM:
For the container management to work, you need to ensure that the VM network can be reached from Dom0. In case you’re not using DHCP in your environment, you need to configure the networking for the VM.
2) Enable Container management
After rebooting into the freshly installed VM, you can enable Container Management using the VM properties in XenCenter.
3) Confirm Container monitoring is working
The easiest way to do this is to confirm that Docker Version information is available in the VM general tab in XenCenter.
You can spin up containers inside the CoreOS VM using the core-user:
docker run --name hello -d busybox /bin/sh -c "while true; do echo Hello World; sleep 1; done"
And you should be able to see and control the containers in the XenCenter tree-view as described in the above.
In case any of the above does not work, check the Notifications and Alerts, which may provide a pointer.
How to monitor other Linux OSs
CoreOS VMs installed with the Cloud Config Drive are automatically prepared for Container Management and the management only needs to be enabled. Other Linux VMs can be prepared manually.
1) Install Docker, ncat and SSHD inside the VM
The installation on Ubuntu can be performed using :
apt-get install docker.io nmap openssh-server
On RHEL/OEL/CentOS 7 the installation can be performed using:
yum install docker nmap-ncat openssh-server
# Enable autostart for docker.service
systemctl enable docker.service
# Start docker.service directly
systemctl start docker.service
A non-root user can be provided access to Docker by adding it to the "Docker" group.
2) Prepare the Container Monitoring of the VM
To prepare a VM, the following command needs to be run in Dom0:
xscontainer-prepare-vm –v [vm-uuid] -u [username]
With being the UUID of the VM, that should be prepared and being the username on the VM that the container management will attempt to use for SSH access.
This preparation script will guide you through the process and automatically enable container management for this VM.
The XenServer team has tested the prepare script and functionality with Ubuntu 14.04 and CentOS/RHEL/OEL 7.0. However other Linux distributions and versions may work as well.
Bootnote - the technical details
So how is a VM actually enabled for Container management? Firstly it needs the VM object in the XAPI database to have the vm:other-config:xscontainer-monitor key set to "true" - this is used by a daemon in domain 0 to know which VMs it should query for the Docker container status. The daemon uses SSH to communicate with managed VMs. A private-public key pair is used to login to managed VMs. While the private key is only known by the daemon of the XenServer pool, the public key is registered as an authorized key in each monitored VM. The username used to login, is specified as vm:other-config:xscontainer-username. This configuration is automatically established for CoreOS VMs created using the XenCenter wizard and for other VMs using the xscontainer-prepare-vm script.
In order for the container management to work, managed VMs have to be reachable via SSH from domain 0. The general networking topology and firewalls must allow an outbound SSH (TCP port 22) connection from domain 0 (i.e. the management network) to “Container Management” VMs. As normal we recommend putting VMs on a different network to XenServer's management network therefore this SSH traffic will likely be routed from domain 0 to the VMs via a firewall external to XenServer. We're considering other inspection mechanisms, that don’t rely on the network, as a potential improvement in the future.