Purpose: Test NBD-based live migration (with 'paused-before-switchover'[*]) using raw JSON QMP commands. https://lists.gnu.org/archive/html/qemu-devel/2017-10/msg04895.html [PATCH v4 0/7] migration: pause-before-switchover Tested with: {"version": {"qemu": {"micro": 90, "minor": 10, "major": 2}, "package": " (v2.11.0-rc0-47-g55ed8d6)"}, "capabilities": []}} (1) [SOURCE] and [DEST]: Launch source and destination QEMU on two different shells, run the `qmp_capabilities` command on source and destination. ----------------------------------------------------------------------- [SOURCE] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $> ~/build/qemu-build/x86_64-softmmu/qemu-system-x86_64 -display none \ -nodefconfig -nodefaults -m 512 -device virtio-scsi-pci,id=scsi \ -device virtio-serial-pci \ -drive file=./src-disk.qcow2,format=qcow2,if=virtio \ -serial unix:/tmp/monitor,server,nowait -qmp stdio { "execute": "qmp_capabilities" } {"return": {}} ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [DEST] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $> qemu-img info src-disk.qcow2 image: src-disk.qcow2 file format: qcow2 virtual size: 39M (41126400 bytes) disk size: 14M cluster_size: 65536 Format specific information: compat: 0.10 refcount bits: 16 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $> qemu-img create -f qcow2 -o preallocation=falloc dst-disk.qcow2 40M Formatting 'dst-disk.qcow2', fmt=qcow2 size=41943040 encryption=off cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $> qemu-img info dst-disk.qcow2 image: dst-disk.qcow2 file format: qcow2 virtual size: 40M (41943040 bytes) disk size: 40M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # Note: we're not starting the destination with '-S'; just "-incoming" # seems to suffice $> ~/build/qemu-build/x86_64-softmmu/qemu-system-x86_64 -display none \ -nodefconfig -nodefaults -m 512 -device virtio-scsi-pci,id=scsi \ -device virtio-serial-pci \ -drive file=./dst-disk.qcow2,format=qcow2,if=virtio \ -serial unix:/tmp/monitor,server,nowait \ -incoming tcp:localhost:6666 -qmp stdio { "execute": "qmp_capabilities" } {"return": {}} ----------------------------------------------------------------------- (2) [SOURCE] Set the "pause-before-switchover" capability. ----------------------------------------------------------------------- {"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"pause-before-switchover","state":true}]}} {"return": {}} ----------------------------------------------------------------------- {"execute":"query-migrate-capabilities"} {"return": [{"state": false, "capability": "xbzrle"}, {"state": false, "capability": "rdma-pin-all"}, {"state": false, "capability": "auto-converge"}, {"state": false, "capability": "zero-blocks"}, {"state": false, "capability": "compress"}, {"state": false, "capability": "events"}, {"state": false, "capability": "postcopy-ram"}, {"state": false, "capability": "x-colo"}, {"state": false, "capability": "release-ram"}, {"state": false, "capability": "block"}, {"state": false, "capability": "return-path"}, {"state": true, "capability": "pause-before-switchover"}, {"state": false, "capability": "x-multifd"}]} ----------------------------------------------------------------------- (3) [DEST] Start the NBD server ----------------------------------------------------------------------- { "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet","data": { "host": "localhost", "port": "3333" } } } } {"return": {}} { "execute": "nbd-server-add", "arguments": { "device": "virtio0","writable": true } } {"return": {}} ----------------------------------------------------------------------- (4) [SOURCE] Execute the `drive-mirror` command ----------------------------------------------------------------------- { "execute": "drive-mirror", "arguments": { "device": "virtio0", "target": "nbd:localhost:3333:exportname=virtio0", "sync": "full","format": "raw", "mode": "existing" } } {"return": {}} {"timestamp": {"seconds": 1510677793, "microseconds": 645286}, "event": "BLOCK_JOB_READY", "data": {"device": "virtio0", "len": 41126400, "offset": 41126400, "speed": 0, "type": "mirror"}} ----------------------------------------------------------------------- (5) [SOURCE] Issue the `migrate` command (and query the migration status): ----------------------------------------------------------------------- {"execute":"migrate","arguments":{"blk":false,"inc":false,"uri":"tcp:localhost:6666"}} {"return": {}} {"timestamp": {"seconds": 1510744913, "microseconds": 630373}, "event": "STOP"} {"execute":"query-migrate"} {"return": {"expected-downtime": 300, "status": "pre-switchover", "setup-time": 2, "total-time": 110111, "ram": {"total": 537403392, "postcopy-requests": 0, "dirty-sync-count": 2, "page-size": 4096, "remaining": 4141056, "mbps": 268.56432, "transferred": 59759367, "duplicate": 115920, "dirty-pages-rate": 72829, "skipped": 0, "normal-bytes": 58601472, "normal": 14307}}} ----------------------------------------------------------------------- (6) [SOURCE] Issue the `block-job-cancel` command ----------------------------------------------------------------------- {"execute":"block-job-cancel","arguments":{"device":"virtio0"} } {"return": {}} {"timestamp": {"seconds": 1510678024, "microseconds": 526240}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "virtio0", "len": 41126400, "offset": 41126400, "speed": 0, "type": "mirror"}} ----------------------------------------------------------------------- (7) [DEST] Stop the NBD server ----------------------------------------------------------------------- {"execute":"nbd-server-stop"} {"return": {}} ----------------------------------------------------------------------- (8) [Optional][*] [DEST] Resume the guest vCPUs ----------------------------------------------------------------------- {"execute":"cont"} {"return": {}} ----------------------------------------------------------------------- (9) [SOURCE] Gracefully terminate the QEMU process (repeat the same on destination.) ----------------------------------------------------------------------- {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": 1510678728, "microseconds": 574318}, "event": "SHUTDOWN", "data": {"guest": false}} ----------------------------------------------------------------------- * * * [*] NOTE: If you started destination QEMU with "-S" (do not start CPU at start up), then you have to issue the QMP 'cont' command on the Detination. We haven't in our case. And Dave Gilbert says: when using "-incoming" without "-S" on the destination, the migration code will restart the CPUs automatically.