pitter patter on the keyboard

GRUB2 is highly customizable and feature-rich, however it is a bit of a nightmare if you configure bootloaders regularly. Unfortunately GRUB 0.97 doesn’t support ext4 and – if you were an idiot like me – you might have set up your new dedicated with an ext4 /boot. Now, you have neither the time nor patience to re-install and need to make Xen go.

First, copy your Dom0 .config and kernel to /boot/. /etc/grub.d/20_linux_xen picks up on the CONFIG_XEN_PRIVILEGED_GUEST=y variable.

# cp /usr/src/linux/arch/x86_64/boot/bzImage /boot/kernel-dom0
# cp /usr/src/linux/.config /boot/config-dom0

Next, remove the executable flag from /etc/grub.d/10_linux or you will end up with three or four entries that boot straight to the Dom0 kernel without the hypervisor.

# chmod -x /etc/grub.d/10_linux

To add kernel boot parameters to your grub.cfg (as generated with grub2-mkconfig and 20_linux_xen) add these lines to your /etc/default/grub file:

GRUB_CMDLINE_LINUX_XEN_REPLACE="max_loop=128"
GRUB_CMDLINE_XEN="dom0_mem=512"

The former will place your boot options on the Dom0 kernel’s module line. The latter will place boot options on the xen.gz hypervisor’s kernel line.

Now, generate your new configuration:

# grub2-mkconfig -o /boot/grub2/grub.cfg

Once the config file has been written, determine the position of the Xen hypervisor entry and update GRUB_DEFAULT in /etc/default/grub.

Install GRUB2 to your MBR(s) as necessary:

# grub2-install /dev/sda 

I just encountered this while making a tar backup on one of my virtual filesystems:

lost page write due to I/O error on xvdb1
end_request: I/O error, dev xvdb1, sector 33707688
Buffer I/O error on device xvdb1, logical block 4213461

Unfortunately, my fs suffered corruption.

Some folks, particularly Ubuntu users with kernel 3.2.x seem to say the answer is adding

barrier=0

to your ext3/ext4 mount options.

In my case I’m pretty sure an OOM error killed something critical to the backend on dom0.

Be sure to check your dom0′s dmesg.

UPDATE Uhh yeah, something actually managed to overflow the dom0′s fs. >.> This is what I was getting in the dom0 dmesg:

loop: Write error at byte offset 17820364800, length 4096.

for which there is very little discussion, but a quick df told me what was going on.

This raw full disk image is a straight-up install of Ubuntu 12.04.1 Server LTS “Precise Pangolin” on EXT3. No packages were selected for installation other than OpenSSH Server. It will boot under full virtualization platforms (QEMU, VirtualBox, VMWare) without modification (except maybe disk image conversion) but the steps below are required to make it play nicely as a Xen Paravirtualized Guest.

db3fb8b154cb05ab704733bba1a6d70e ubuntu-12.04.1-x86_64.raw.hdd.lzma

The user account is “user”
The user password is “user”

The default network settings are: eth0 DHCP

Fortunately, the generic kernel has Xen PV guest support so the required buggery is mostly limited to making the console work. Unpack and mount the image:

# unlzma ubuntu-12.04.1-x86_64.raw.hdd.lzma
# mkdir /mnt/rawroot
# lomount -diskimage ubuntu-12.04.1-x86_64.raw.hdd -partition 1 /mnt/rawroot

The distributed /boot/grub/grub.cfg script will make pygrub throw up. We’ll drop this (/mnt/rawroot)/boot/grub/grub.conf file in to override it:

default 0
timeout 5
fallback 1

title Ubuntu, with Linux 3.2.0-29-generic
        root=(hd0,0)
        kernel   /boot/vmlinuz-3.2.0-29-generic root=UUID=a52e9498-44e8-4c0d-807e-903d4e19e204 ro console=hvc0
        initrd  /boot/initrd.img-3.2.0-29-generic

title Ubuntu, with Linux 3.2.0-29-generic (recovery mode)
        root=(hd0,0)
        kernel   /boot/vmlinuz-3.2.0-29-generic root=UUID=a52e9498-44e8-4c0d-807e-903d4e19e204 ro recovery nomodeset
        initrd  /boot/initrd.img-3.2.0-29-generic

Now we need to make a getty load on hvc0, create (/mnt/rawroot)/etc/init/hvc0.conf

# hvc0 - getty
#
# This service maintains a getty on hvc0 from the point the system is
# started until it is shut down again.

start on stopped rc or RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
exec /sbin/getty -L 115200 hvc0 vt102

We probably want to give our VM a static IP. Edit (/mnt/rawroot)/etc/network/interfaces to reflect:

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8

Now we are ready to boot up. Unmount /mnt/rawroot and create a configuration file that looks similar to:

name = "ubuntu"

vcpus = 1
memory = 256
vif = [ 'mac=00:16:3e:ff:00:01,bridge=extbr0' ]
disk = ['file:/xen/ubuntu/ubuntu.hdd,sda,w']

bootloader = "/usr/bin/pygrub"
extra = "xencons=hvc0 console=hvc0"

Ensure the MAC address does not conflict with any existing virtual machines. Assuming the configuration file is named ubuntu.conf run

# xm create ubuntu.conf -c

If everything went well you should be looking at a login prompt shortly after.

Note that if you use cp -ax to transfer the filesystem’s contents to a larger image in the future, references using UUIDs in fstab and grub.conf will have to be changed to either reflect the new UUID or the corresponding /dev node (probably xvda1) and GRUB must be re-installed to the MBR.

If you would like to move the contents to a filesystem-only image (no partition table, MBR, etc.) which is much easier to grow than a full disk image you can drop pygrub and externalize the kernel and initrd images then reference them in the configuration file.