Purpose: The below can be useful in any scenario where you'd like to know the interactions between libvirt and QEMU. But the below talks in the context of a live migration bug. If possible to perform the live migration test again, some log files that can be useful. Gather Nova debug logs ---------------------- (1) Sanitized Nova debug logs (preferably contextual on the source and destination hosts): compute.log, conductor.log, scheduler.log To enable Nova debug logs, add these config attributes: . . . verbose = True debug = True . . . Restart Nova services: $ openstack-service restart nova (2) /var/log/libvirt/qemu/instance.log -- For both compute nodes involved. This will give us the QEMU CLI that's being used by libvirt. Gathering libvirt logs with useful log filters ---------------------------------------------- (3) If the test can be repeated, it'll be useful to enable debug logging for libvirt on the Compute nodes involved. As libvirt is the component that's doing the heavy lifting under the hood. These logs gives us detailed interactions between libvirt and QEMU on source and destination Compute nodes. There are two ways to configure libvirt debug logs: (a) manually[+], that requires a libvirtd restart; and (b) dynamically, where a libvirtd restart is not required. Here is how to configure libvirt log filters dynamically: a. Install the `virt-admin` tool, provided by 'libvirt-admin' package: $> dnf install libvirt-admin -y b. Configure the libvirt log file: $> virt-admin daemon-log-outputs "1:file:/var/log/libvirt/libvirtd.log" c. Configure the most useful log filters. (They also let you observe the communication between libvirt and QEMU -- e.g. QMP commands libvirt sends to QEMU. They are useful to debug bugs involving interactions between libvirt and QEMU): $> virt-admin daemon-log-filters \ "1:libvirt 1:qemu 1:conf 1:security 3:event 3:json 3:file 3:object 1:util" Ensure that the filters are actually applied by running `virt-admin daemon-log-filters` (with no extra args), which should show the log filters that are currently applied. That's it; no need to restart the libvirt daemon. The /var/log/libvirt/libvirtd.log will now start capturing the debug logs with the applied filters. d. Now, repeat the test. e. Then attach the libvirtd.log (from all relevant hosts) as plain text to the bug. NOTE: Do *not* forget to turn off both the Nova and libvirt debugging controls once the test is finished -- otherwise, it continues to log heavily and can fill up the disk space quickly. To turn off the dynamic log filters from the command-line: $> virt-admin daemon-log-filters "" Check that the filters are actually removed, by running `virt-admin daemon-log-filters` (with no extra args), which should show an empty list. * * * [+] To manually enable libvirt log filters ------------------------------------------ This is an alternative approach to using the `virt-admin` tool: 1. In /etc/libvirt/libvirtd.conf, have these two config attributes: . . . log_filters="1:libvirt 1:qemu 1:conf 1:security 3:event 3:json 3:file 3:object 1:util" log_outputs="1:file:/var/log/libvirt/libvirtd.log" . . . 2. Restart libvirtd: $ systemctl restart libvirtd (Then, repeat the test and attach the libvirtd.log from all relevant hosts as plain text to the bug.)