13-Aug-2011 Reference: Follow https://lwn.net/Articles/447556/ == TEST ENV. == Host: Fedora -15 Processor: 4 Intel(R) Xeon(R) CPU E5405 @ 2.00GHz RAM: 8G == PROCEDURE == + Install all the pre-reqs. on the host (Fedora-15) - Ensure to have these packages -- gcc, glibc-devel(both arches) make - yum install gcc glibc-devel glibc-devel-2.14-5.i686 make + Clone the repository. - git clone git://github.com/penberg/linux-kvm.git (#note this takes a little time, if there isn't already a local tree w/ you) ################################ [root@moon kvm]# pwd /export/native-linux-kvm/linux-kvm/tools/kvm [root@moon kvm]# ################################ [root@moon kvm]# make CC bios/e820.o CC bios/int10.o CC bios/int15.o CC bios/entry.o LD bios/bios.bin.elf OBJCOPY bios/bios.bin NM bios/bios-rom.h CC bios.o CC bios/bios-rom.o LINK kvm [root@moon kvm]# ################################ [root@moon test-images]# pwd /export/test-images [root@moon test-images]# ################################ [root@moon test-images]# wget -c http://wiki.qemu.org/download/linux-0.2.img.bz2 && bunzip2 linux-0.2.img.bz2 [root@moon test-images]# ls linux-0.2.img [root@moon test-images]# ################################ + Now, buiding the kernel - Ensure to have ncurses libraries installed - # yum install ncurses* (for 'make menuconfig' to run successfully) - type 'make menuconfig' - In the ncurses window, search (/) for the configure options, and then naviate to that path in the 'make menuconfig' ncurses window. These are the kernel options we need. ####################################################################### [root@moon linux-kvm]# find . -name *config* | xargs egrep -i 'VIRTIO_BLK|VIRTIO_NET|VIRTIO_CONSOLE|SERIAL_8250_CONSOLE|HW_RANDOM_VIRTIO|FB_VESA|CONFIG_EXT4_FS' ./arch/mips/Kconfig: select SERIAL_8250_CONSOLE ./arch/arm/mach-omap2/Kconfig: select SERIAL_8250_CONSOLE ./arch/arm/mach-omap2/Kconfig: select SERIAL_8250_CONSOLE ./arch/x86/lguest/Kconfig: select VIRTIO_CONSOLE ./arch/s390/Kconfig: select VIRTIO_CONSOLE ./drivers/video/Kconfig:config FB_VESA ./drivers/tty/serial/Kconfig:config SERIAL_8250_CONSOLE ./drivers/net/Kconfig:config VIRTIO_NET ./drivers/char/hw_random/Kconfig:config HW_RANDOM_VIRTIO ./drivers/char/Kconfig:config VIRTIO_CONSOLE ./drivers/block/Kconfig:config VIRTIO_BLK ./drivers/vhost/Kconfig: guest networking with virtio_net. Not to be confused with virtio_net [root@moon linux-kvm]# ####################################################################### + Kernel Configuration: CONFIG_VIRTIO_BLK - Prompt: Virtio block driver(EXPERIMENTAL) - Navigate to this path: Device Drivers -> Block Devices -> Virtio block driver - Select 'y' to include CONFIG_VIRTIO_NET - Prompt: Virtio network driver(EXPERIMENTAL) - Navigate to this path: Device Drivers -> Network device support -> Virtio network driver - Select 'y' to include CONFIG_VIRTIO_CONSOLE - Prompt: Virtio console - Navigate to this path: Device Drivers -> Character devices -> Virtio console - Select 'y' to include CONFIG_SERIAL_8250_CONSOLE - Prompt: Console on 8250/16550 and compatible serial port - Navigate to this path: Device Drivers -> Character devices -> Serial drivers -> 8250/16550 and compatible serial support - Select 'y' to include (or ensure that this is already included. * indicates it's included ) CONFIG_HW_RANDOM_VIRTIO - Prompt: VirtIO Hardware Random Number Generator Core support - Navigate to this path: Device Drivers -> Character devices -> Hardware Random Number Generator Core support -> Select the sub-option "VirtIO Random Number Generator support" - Select 'y' to include CONFIG_FB_VESA - Prompt: VESA VGA graphics support - Navigate to this path: Device Drivers -> Graphics support -> Support for frame buffer devices (FB [=y]) - Select 'y' to include (or ensure that this is already included. * indicates it's included ) CONFIG_EXT4_FS - Prompt: VESA VGA graphics support - Navigate to this path: Device Drivers -> Graphics support -> Support for frame buffer devices (FB [=y]) - Select 'y' to include (or ensure that this is already included. * indicates it's included ) Exit and SAVE the configuration. Now, grep the '.config' file in the linux-kvm to ensure all the appropriate configurations are selected for native-linux-tools. #################################################################### [root@moon linux-kvm]# egrep 'VIRTIO_BLK|VIRTIO_NET|VIRTIO_CONSOLE|SERIAL_8250_CONSOLE|HW_RANDOM_VIRTIO|FB_VESA|CONFIG_EXT4_FS|VIRTIO_PCI' .config CONFIG_VIRTIO_BLK=y CONFIG_VIRTIO_NET=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_VIRTIO_CONSOLE=y CONFIG_HW_RANDOM_VIRTIO=y CONFIG_FB_VESA=y CONFIG_EXT4_FS=y CONFIG_EXT4_FS_XATTR=y CONFIG_EXT4_FS_POSIX_ACL=y CONFIG_EXT4_FS_SECURITY=y CONFIG_VIRTIO_PCI=y [root@moon linux-kvm]# #################################################################### Also ensure the CONFIG_VIRTIO_PCI=y. + Also for the guest kernel, Sashal suggested to compile the guest kernel without any modules. - Reason: If you build it with modules you'll need to copy the modules into the image before booting the kernel. If we have all of them built in, we just use the bzImage. ################################################################# [root@moon linux-kvm]# sed -i 's/=m/=y/' .config [root@moon linux-kvm]# ################################################################# + Then compile by just running 'make' (go w/ the defaults, just hit 'Enter') ################################################################# [root@moon linux-kvm]# make ################################################################# + Since, it was running slow, I aborted the above make)it'll save the progress it did so far). and ran it with '-j5' option to make. This will build it with 5 threads in parallel. ############################ [root@moon linux-kvm]# make -j5 ############################ + So note, before trying the qcow2 image, we have to first de-compress it. (use the below syntax- thanks to sashal and penberg) - Other notes: penberg mentioned that this qcow2 image was a 'RAW' image converted to 'Qcow2' - To decompress it, (I'm also using the option 'preallocation=metadata' to get good performance out of the Qcow2 image. ################################################################# [root@moon test-images]# qemu-img convert -f qcow2 -O qcow2 -o preallocation=metadata debian_squeeze_i386_standard.qcow2 debian_squeeze_i386_standard_decompressed.qcow2 ############################ [root@moon test-images]# ls debian_squeeze_i386_standard_decompressed.qcow2 debian_squeeze_i386_standard.qcow2 linux-0.2.img [root@moon test-images]# ################################################################# - A successful compilation should result in generating the 'bzImage' as below: ################################################################# . . . Root device is (8, 3) Setup is 16860 bytes (padded to 16896 bytes). System is 20877 kB CRC 96b004c8 Kernel: arch/x86/boot/bzImage is ready (#1) [root@moon linux-kvm]# ################################################################# == OBSERVATIONS WHILE BOOTING == + Now, let's try to launch the hypervisor boot a few different images: 1/ boot into the bzImage. (- this will boot into the host rootfs (apparently). All I can see is exiting - [root@moon kvm]# ./kvm run -k ../../arch/x86/boot/bzImage #################################### . . . [ 24.630496] sky2: driver version 1.28 [ 24.630747] D-Link DE-620 pocket adapter io 0x378, which is busy. [ 24.630889] QLogic 1/10 GbE Converged/Intelligent Ethernet Driver v5.0.18 [ 24.631064] PPP generic driver version 2.4.2 [ 24.631153] PPP Deflate Compression module registered [ 24.734503] sd 0:0:0:0: Attached scsi generic sg0 type 0 [ 24.735066] sd 0:0:0:0: [sda] 16384 512-byte logical blocks: (8.38 MB/8.00 MiB) [ 24.735441] PPP MPPE Compression module registered [ 24.735446] NET: Registered protocol family 24 [ 24.735468] PPTP driver version 0.8.5 [ 24.736076] sd 0:0:0:0: [sda] Write Protect is off [ 24.736216] SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256). [ 24.736218] CSLIP: code [root@moon kvm]# #################################### 2/ boot into the linux-0.2.img (minimal raw user-space image) root@moon kvm]# ./kvm run -d /export/test-images/linux-0.2.img - Observations: 1] Firstly, it doesn't boot into the image, it just exits. 2] If you boot (immediately, in quick succession twice, you see the below error. Sashal also noticed this on his laptop, assuming this is a hardware issue) #################################### [root@moon kvm]# ./kvm run -d /export/test-images/linux-0.2.img # kvm run -k ../../arch/x86/boot/bzImage -m 448 -c 4 --name guest-9408 KVM_GET_SUPPORTED_CPUID failed: Argument list too long #################################### 3] Exits abruptly and ungracefully ( attempting to boot the minimal RAW guest image from qemu.org) ##################################### . . . [ 63.860249] bits in page size: 9 [ 63.860250] bits in OOB size: 4 [ 63.860252] flash size with OOB: 135168 KiB [ 63.860275] page address bytes: 4 [ 63.860276] sector address bytes: 3 [ 63.860278] options: 0x62 [ 63.863290] Scanning device for bad blocks [ 63.889433] Creating 1 MTD partitions on "NAND 128MiB 1,8V 8-bit": [ 63.889439] 0x000000000000-0x000008000000 : "NAND simulator partition 0" [ 63.889897] ftl_cs: FTL header not found. [ 63.902839] usbcore: registered new interface driver alauda [ 63.903533] Fixed MDIO Bus: probed [ 63.903599] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI [ 63.903601] e1000: Copyright (c) 1999-2006 Intel Corporation. [ 63.903657] e1000e: Intel(R) PRO/1000 Network Driver - 1.3.10-k2 [ 63.903659] e1000e: Copyright(c) 1999 - 2011 Intel Corporation. [ 63.903708] Intel(R) Gigabit Ethernet Network Driver - version 3.0.6-k2 [ 63.903710] Copyright (c) 2007-2011 Intel Corporation. [ 63.903732] Intel(R) Virtual Function Network Driver - version 1.0.8-k0 [ 63.903734] Copyright (c) 2009 - 2010 Intel Corporation. [ 63.903753] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.3.8-k2 [ 63.903755] ixgbe: Copyright (c) 1999-2011 Intel Corporation. [ 63.903774] ixgbevf: Intel(R) 10 Gigabit PCI Exp[root@moon kvm]# ##################################### + Notes from IRC discussion with Sasha Levin: (thanks a lot helping me along) - What's strange is, once the just built kernel(located at arch/x86/boot/bzImage finishes booting, it should start running the image I provide (in this case the linux-02.img downloaded from qemu.org --( wget http://wiki.qemu.org/download/linux-0.2.img.bz2 && bunzip2 linux-0.2.img.bz2). But in this case, it is just randomly exiting half-way through loading the kernel. - It is supposed to fail a meaningful error of why it's failing (And this also doesn't go through the regular exit path). And..... Sasha was kind enough to send me his kernel .config. I recompiled and ran the linux-0.2.img from qemu.org ######################################## [root@moon tools]# ./kvm run -d /export/test-images/linux-0.2.img | tee /var/tmp/linux-0.2.img-booting.txt ######################################## Wooo..... it boots: ######################################## [ 1.377909] rtc0: alarms up to one day, 114 bytes nvram [ 1.379046] i2c /dev entries driver [ 1.380000] Linux video capture interface: v2.00 [ 1.381718] coretemp coretemp.0: Adding Core 0 failed [ 1.382836] coretemp coretemp.0: Adding Core 1 failed [ 1.384338] coretemp coretemp.2: Adding Core 2 failed [ 1.385813] coretemp coretemp.3: Adding Core 3 failed [ 1.386747] Bluetooth: Virtual HCI driver ver 1.3 [ 1.387838] Bluetooth: HCI UART driver ver 2.2 [ 1.388615] Bluetooth: HCI H4 protocol initialized [ 1.389481] Bluetooth: HCI BCSP protocol initialized [ 1.390366] Bluetooth: HCILL protocol initialized sh-2.05b# ######################################## sh-2.05b# ls sh-2.05b# pwd /export sh-2.05b# echo test > foo.txt sh-2.05b# cat foo.txt test sh-2.05b# ls / bin dev export lost+found proc sbin usr boot etc lib mnt root tmp var sh-2.05b# ######################################## Exit:(I believe this is graceful) ######################################## sh-2.05b# exit exit [ 994.456165] EXT4-fs (vda): re-mounted. Opts: (null) [ 996.470293] Unregister pv shared memory for cpu 2 [ 996.470305] Unregister pv shared memory for cpu 1 [ 996.470337] Unregister pv shared memory for cpu 3 [ 996.472699] Unregister pv shared memory for cpu 0 [ 996.474410] Restarting system. [ 996.474945] machine restart # KVM session ended normally. [root@moon kvm]# ########################################