Tracing KVM with ftrace ======================= Doc --- http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/trace/tracepoint-analysis.txt?id=HEAD http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/trace/ftrace.txt?id=HEAD http://lwn.net/Articles/370423/ http://lwn.net/Articles/365835/ http://lwn.net/Articles/366796/ Tracing L2 guest ---------------- $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git trace-cmd # Build and Install $ make && sudo make-install # Optionally $ sudo yum install asciidoc -y $ sudo make install_doc # To find all possible events $ sudo find /sys/kernel/debug/tracing/events -type d # Let's find what's relevant for KVM $ sudo find /sys/kernel/debug/tracing/events -type d /sys/kernel/debug/tracing/events/kvmmmu # Start Trace command with kvmmmu $ trace-cmd record -b 2000 -e kvmmmu /sys/kernel/debug/tracing/events/kvmmmu/filter /sys/kernel/debug/tracing/events/*/kvmmmu/filter Hit Ctrl^C to stop recording (trace-cmd will wait) Run your workload, then stop trace-cmd with ctrl-C. trace-cmd will write a trace.dat file. # Start L2 guest $ virsh start nguest-01 --console After a minute or two, ctl-C the trace-cmd, # Generate the report $ trace-cmd report More tracing ------------ This works: $ trace-cmd record -e kvm:kvm_exit -e kvm:kvm_entry \ -e kvm:kvm_nested_vmexit -e kvm:kvm_nested_vmrun Tips (Thanks Paolo Bonzini) -------------------------- $ trace-cmd record -e kvm/* qemu-system-x86_64 [. . .] $ trace-cmd record -e kvm -e kvmmu -P 18697"