=^.^=

Migrate a Virtual Machine from Full Disk to Single File-System Image

karma

Full-disk images - that is to say a file containing an MBR, partition table and one or more partitions - are a pain in the arse to scale. Conventional wisdom tells us we should be able to do this with losetup and parted, or less gracefully with (c)fdisk but I've never had anything but trouble. In the end, it tends to be much easier to move the contents of the image to a file-system-only image and externalize the kernel since we'll no longer have the benefit of a bootloader.

Let's create and mount our new image to start off:

# dd if=/dev/zero of=newimage.img bs=1M count=4000
# mke2fs -j newimage.img
# mkdir /mnt/rawroot
# mount -o loop newimage.img /mnt/rawroot

Now we're going to mount our old full disk image's partition. Use kpartx -l to determine what node the partition will be mapped to:

# kpartx -l oldimage.hdd
loop34p1 : 0 4190208 /dev/loop34 2048

This tells us the full disk will be on /dev/loop34 and the partition itself will be on /dev/mapper/loop34p1.

# kpartx -a oldimage.hdd
# mkdir /mnt/rawroot2
# mount /dev/mapper/loop34p1 /mnt/rawroot2

Alternatively, we can let lomount do the work for us:

# lomount -diskimage oldimage.hdd -partition 1 /mnt/rawroot2

Now we'll copy the files in the old file system to the new image exactly as they stand:

# cd /mnt/
# cp -ax rawroot2/* rawroot/

It's important that we use the -a flag or we may end up breaking file system links and will almost certainly fill up the new image when we hit device nodes like /dev/zero, /dev/urandom and so on.

Edit the /etc/fstab in the new image and ensure it reflects the new configuration; / will probably come from /dev/xvda1. Old references by LABEL or UUID will no longer be valid.

Since we'll be forgoing a bootloader it is necessary to externalize the kernel and initrd images:

# cp rawroot2/boot/vmlinuz-3.2.0-29-generic /xen/myvm/
# cp rawroot2/boot/initrd.img-3.2.0-29-generic /xen/myvm/

Now we can unmount both images:

# umount rawroot
# umount rawroot2

If you used kpartx instead of lomount you should remove the old image from the loop device:

# kpartx -d oldimage.hdd

Now alter the Xen configuration file to reference the externalized kernel and initrd:

kernel = "/xen/myvm/vmlinuz-3.2.0-29-generic"
initrd = "/xen/myvm/initrd.img-3.2.0-29-generic"

Remove or comment your reference to pygrub and boot the machine. If everything went well you will be looking at a login prompt shortly.

Please see my article on resizing file system images for details on how to grow the new image.

Force a User to Log Out

karma

It looks like I have a stale session open which is preventing me from umounting a disk image:

# w
 13:21:37 up 56 days,  2:51,  2 users,  load average: 4.30, 3.70, 2.23
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
kfox     pts/8    192.168.8.90     Wed13   20:49m  0.02s  0.02s -bash
kfox     pts/22   192.168.8.90     12:04    1.00s  0.07s  0.00s w

Let's find the process responsible for my session (the "session leader"):

# ps -dN | grep pts/8
20524 pts/8    00:00:00 bash

Now we'll kill the bugger:

# kill -9 20524

How to Tell What Version of a RedHat-Based Flavour you are Using

karma

It's important to know what version of RHEL/Fedora/CentOS/Scientific Linux/etc. you are dealing with when looking for version-compatible RPMs that are out-of-repo. I always end up forgetting how to do this, so for our mutual benefit:

$ cat /etc/redhat-release
CentOS release 5.6 (Final)

64-bit Ubuntu 12.04.1 Server Virtual Machine Image for Xen, QEMU, VirtualBox etc.

karma

This raw full disk image is a straight-up install of Ubuntu 12.04.1 Server LTS "Precise Pangolin" on EXT3. No packages were selected for installation other than OpenSSH Server. It will boot under full virtualization platforms (QEMU, VirtualBox, VMWare) without modification (except maybe disk image conversion) but the steps below are required to make it play nicely as a Xen Paravirtualized Guest.

db3fb8b154cb05ab704733bba1a6d70e ubuntu-12.04.1-x86_64.raw.hdd.lzma

The user account is "user"
The user password is "user"

The default network settings are: eth0 DHCP

Fortunately, the generic kernel has Xen PV guest support so the required buggery is mostly limited to making the console work. Unpack and mount the image:

# unlzma ubuntu-12.04.1-x86_64.raw.hdd.lzma
# mkdir /mnt/rawroot
# lomount -diskimage ubuntu-12.04.1-x86_64.raw.hdd -partition 1 /mnt/rawroot

The distributed /boot/grub/grub.cfg script will make pygrub throw up. We'll drop this (/mnt/rawroot)/boot/grub/grub.conf file in to override it:

default 0
timeout 5
fallback 1

title Ubuntu, with Linux 3.2.0-29-generic
        root=(hd0,0)
        kernel   /boot/vmlinuz-3.2.0-29-generic root=UUID=a52e9498-44e8-4c0d-807e-903d4e19e204 ro console=hvc0
        initrd  /boot/initrd.img-3.2.0-29-generic

title Ubuntu, with Linux 3.2.0-29-generic (recovery mode)
        root=(hd0,0)
        kernel   /boot/vmlinuz-3.2.0-29-generic root=UUID=a52e9498-44e8-4c0d-807e-903d4e19e204 ro recovery nomodeset
        initrd  /boot/initrd.img-3.2.0-29-generic

Now we need to make a getty load on hvc0, create (/mnt/rawroot)/etc/init/hvc0.conf

# hvc0 - getty
#
# This service maintains a getty on hvc0 from the point the system is
# started until it is shut down again.

start on stopped rc or RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
exec /sbin/getty -L 115200 hvc0 vt102

We probably want to give our VM a static IP. Edit (/mnt/rawroot)/etc/network/interfaces to reflect:

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8

Now we are ready to boot up. Unmount /mnt/rawroot and create a configuration file that looks similar to:

name = "ubuntu"

vcpus = 1
memory = 256
vif = [ 'mac=00:16:3e:ff:00:01,bridge=extbr0' ]
disk = ['file:/xen/ubuntu/ubuntu.hdd,sda,w']

bootloader = "/usr/bin/pygrub"
extra = "xencons=hvc0 console=hvc0"

Ensure the MAC address does not conflict with any existing virtual machines. Assuming the configuration file is named ubuntu.conf run

# xm create ubuntu.conf -c

If everything went well you should be looking at a login prompt shortly after.

Note that if you use cp -ax to transfer the filesystem's contents to a larger image in the future, references using UUIDs in fstab and grub.conf will have to be changed to either reflect the new UUID or the corresponding /dev node (probably xvda1) and GRUB must be re-installed to the MBR.

If you would like to move the contents to a filesystem-only image (no partition table, MBR, etc.) which is much easier to grow than a full disk image you can drop pygrub and externalize the kernel and initrd images then reference them in the configuration file.

Documentary for Dinner: The Vice Guide to the Balkans (2012)

karma

VICE takes us on a quick tour of former Yugoslavian states Serbia, Kosovo and Bosnia to investigate long standing ethnic tensions