Calculate a common baseline CPU

Calculate a common baseline CPU from different hosts

The below shows the method to calculate a common CPU baseline from two different hosts. This baseline CPU can then be provided as a vCPU for guests running on both hosts; it will allow live migration between both hosts.

This example is based on two CPU hosts:

First host: Intel(R) Core(TM) i7-8650U CPU
Second hosts: Intel(R) Xeon(R) CPU E5-2620 v2

The rough procedure here is to run the below command on both (or however many there are) compute hosts:

$> virsh domcapabilities | xmllint --xpath "//cpu/mode[@name='host-model']" -

Then, open a file and copy/paste the stuff that is under <mode> ... </mode> into <cpu> ... </cpu>. (This is a libvirt quirk to get the input parsed correctly by the 'hypervisor-cpu-baseline' API.) Now repeat the same for other hosts, and append each of the <cpu> ... </cpu> output into the same file. Then provide this file as an input to virsh hypervisor-cpu-baseline to get a common guest CPU that can be given to guests on the involved hosts.

Example

Run the 'domcapabilities' command on the first host:

$> virsh domcapabilities | xmllint --xpath "//cpu/mode[@name='host-model']" -
<mode name="host-model" supported="yes">
      <model fallback="forbid">Skylake-Client-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy="require" name="ss"/>
      <feature policy="require" name="vmx"/>
      <feature policy="require" name="pdcm"/>
      <feature policy="require" name="hypervisor"/>
      <feature policy="require" name="tsc_adjust"/>
      <feature policy="require" name="clflushopt"/>
      <feature policy="require" name="umip"/>
      <feature policy="require" name="md-clear"/>
      <feature policy="require" name="stibp"/>
      <feature policy="require" name="arch-capabilities"/>
      <feature policy="require" name="ssbd"/>
      <feature policy="require" name="xsaves"/>
      <feature policy="require" name="pdpe1gb"/>
      <feature policy="require" name="invtsc"/>
      <feature policy="require" name="ibpb"/>
      <feature policy="require" name="ibrs"/>
      <feature policy="require" name="amd-stibp"/>
      <feature policy="require" name="amd-ssbd"/>
      <feature policy="require" name="rsba"/>
      <feature policy="require" name="skip-l1dfl-vmentry"/>
      <feature policy="require" name="pschange-mc-no"/>
      <feature policy="disable" name="hle"/>
      <feature policy="disable" name="rtm"/>
    </mode>

Open a file, "domCaps-CPUs.xml", and copy/paste the output from under <mode> ... </mode> under <cpu> ... </cpu> (i.e. we're removing <mode> element):

$> cat domCaps-CPUs.xml
<cpu>
      <model fallback="forbid">Skylake-Client-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy="require" name="ss"/>
      <feature policy="require" name="pdcm"/>
      <feature policy="require" name="hypervisor"/>
      <feature policy="require" name="tsc_adjust"/>
      <feature policy="require" name="clflushopt"/>
      <feature policy="require" name="umip"/>
      <feature policy="require" name="md-clear"/>
      <feature policy="require" name="stibp"/>
      <feature policy="require" name="arch-capabilities"/>
      <feature policy="require" name="ssbd"/>
      <feature policy="require" name="xsaves"/>
      <feature policy="require" name="pdpe1gb"/>
      <feature policy="require" name="invtsc"/>
      <feature policy="require" name="ibpb"/>
      <feature policy="require" name="ibrs"/>
      <feature policy="require" name="amd-stibp"/>
      <feature policy="require" name="amd-ssbd"/>
      <feature policy="require" name="rsba"/>
      <feature policy="require" name="skip-l1dfl-vmentry"/>
      <feature policy="require" name="pschange-mc-no"/>
      <feature policy="disable" name="hle"/>
      <feature policy="disable" name="rtm"/>
</cpu>

Now, run the 'domcapabilities' on the second host (notice, the output here slightly different from what we have in step-1 above):

$> virsh domcapabilities | xmllint --xpath "//cpu/mode[@name='host-model']" -
<mode name="host-model" supported="yes">
      <model fallback="forbid">IvyBridge-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy="require" name="ss"/>
      <feature policy="require" name="vmx"/>
      <feature policy="require" name="pdcm"/>
      <feature policy="require" name="pcid"/>
      <feature policy="require" name="hypervisor"/>
      <feature policy="require" name="arat"/>
      <feature policy="require" name="tsc_adjust"/>
      <feature policy="require" name="umip"/>
      <feature policy="require" name="md-clear"/>
      <feature policy="require" name="stibp"/>
      <feature policy="require" name="arch-capabilities"/>
      <feature policy="require" name="ssbd"/>
      <feature policy="require" name="xsaveopt"/>
      <feature policy="require" name="pdpe1gb"/>
      <feature policy="require" name="invtsc"/>
      <feature policy="require" name="ibpb"/>
      <feature policy="require" name="amd-ssbd"/>
      <feature policy="require" name="skip-l1dfl-vmentry"/>
      <feature policy="require" name="pschange-mc-no"/>
    </mode>

Here too, copy/paste the output from <mode> ... </mode> under a new <cpu> ... </cpu> element in the _same_ file as in step-2 above.

So the "domCaps-CPUs.xml" with two <cpu> ... </cpu> elements appended next to each other looks as below:

$> cat domCaps-CPUs.xml
<cpu>
      <model fallback="forbid">Skylake-Client-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy="require" name="ss"/>
      <feature policy="require" name="vmx"/>
      <feature policy="require" name="pdcm"/>
      <feature policy="require" name="hypervisor"/>
      <feature policy="require" name="tsc_adjust"/>
      <feature policy="require" name="clflushopt"/>
      <feature policy="require" name="umip"/>
      <feature policy="require" name="md-clear"/>
      <feature policy="require" name="stibp"/>
      <feature policy="require" name="arch-capabilities"/>
      <feature policy="require" name="ssbd"/>
      <feature policy="require" name="xsaves"/>
      <feature policy="require" name="pdpe1gb"/>
      <feature policy="require" name="invtsc"/>
      <feature policy="require" name="ibpb"/>
      <feature policy="require" name="ibrs"/>
      <feature policy="require" name="amd-stibp"/>
      <feature policy="require" name="amd-ssbd"/>
      <feature policy="require" name="rsba"/>
      <feature policy="require" name="skip-l1dfl-vmentry"/>
      <feature policy="require" name="pschange-mc-no"/>
      <feature policy="disable" name="hle"/>
      <feature policy="disable" name="rtm"/>
</cpu>
<cpu>
      <model fallback="forbid">IvyBridge-IBRS</model>
      <vendor>Intel</vendor>
      <feature policy="require" name="ss"/>
      <feature policy="require" name="vmx"/>
      <feature policy="require" name="pdcm"/>
      <feature policy="require" name="pcid"/>
      <feature policy="require" name="hypervisor"/>
      <feature policy="require" name="arat"/>
      <feature policy="require" name="tsc_adjust"/>
      <feature policy="require" name="umip"/>
      <feature policy="require" name="md-clear"/>
      <feature policy="require" name="stibp"/>
      <feature policy="require" name="arch-capabilities"/>
      <feature policy="require" name="ssbd"/>
      <feature policy="require" name="xsaveopt"/>
      <feature policy="require" name="pdpe1gb"/>
      <feature policy="require" name="invtsc"/>
      <feature policy="require" name="ibpb"/>
      <feature policy="require" name="amd-ssbd"/>
      <feature policy="require" name="skip-l1dfl-vmentry"/>
      <feature policy="require" name="pschange-mc-no"/>
</cpu>

Now calculate the CPU baseline by passing the "domCaps-CPUs.xml" as an input to 'hypervisor-cpu-baseline' command:

$> virsh hypervisor-cpu-baseline domCaps-CPUs.txt
<cpu mode='custom' match='exact'>
  <model fallback='forbid'>IvyBridge-IBRS</model>
  <vendor>Intel</vendor>
  <feature policy='require' name='ss'/>
  <feature policy='require' name='vmx'/>
  <feature policy='require' name='pdcm'/>
  <feature policy='require' name='pcid'/>
  <feature policy='require' name='hypervisor'/>
  <feature policy='require' name='arat'/>
  <feature policy='require' name='tsc_adjust'/>
  <feature policy='require' name='umip'/>
  <feature policy='require' name='md-clear'/>
  <feature policy='require' name='stibp'/>
  <feature policy='require' name='arch-capabilities'/>
  <feature policy='require' name='ssbd'/>
  <feature policy='require' name='xsaveopt'/>
  <feature policy='require' name='pdpe1gb'/>
  <feature policy='require' name='invtsc'/>
  <feature policy='require' name='ibpb'/>
  <feature policy='require' name='amd-ssbd'/>
  <feature policy='require' name='skip-l1dfl-vmentry'/>
  <feature policy='require' name='pschange-mc-no'/>
</cpu>

This is the baseline CPU that you can give to guests (by editing the guest XML) on both hosts. This will now allow live migration to happen between both the hosts.

Configuring in OpenStack Nova

You can then configure the Nova compute nodes with the above baseline CPU model and all the CPU features as below. This will allow the guests to be live-migrated between the said hosts:

[libvirt]
cpu_mode = custom
cpu_model = IvyBridge-IBRS
cpu_model_extra_flags="ss,vmx,pdcm,pcid,hypervisor,arat,tsc_adjust,umip,md-clear,stibp,arch-capabilities,ssbd,xsaveopt,pdpe1gb,invtsc,ibpb,amd-ssb,skip-l1dfl-vmentry,pschange-mc-no"

(Scroll horizontally to see the full list of extra flags.)

NOTE: In case the virsh hypervisor-cpu-baseline ouput contains a feature with policy='disable' (as opposed to 'require'), then you can disable that flag in Nova by prefixing "-" before the flag. If you don't specify "+" or "-", Nova will assume the flag is to be enabled.

See examples here in the documentation here: https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cpu_model_extra_flags