Re: Live migration regarding Intel PT

[Eduardo Habkost]

On Thu, Aug 26, 2021 at 10:16:26AM +0800, Xiaoyao Li wrote:
> On 8/26/2021 4:48 AM, Eduardo Habkost wrote:
> > On Wed, Aug 25, 2021 at 02:59:37PM +0800, Xiaoyao Li wrote:
> > > Hi Eduardo,
> > > 
> > > I have some question regrading Intel PT live migration.
> > > 
> > > Commit "e37a5c7fa459 (i386: Add Intel Processor Trace feature support)"
> > > expose Intel PT with a fixed capabilities of CPUID 0x14 for live 
> > > migration.
> > > And the fixed capabilities are the value reported on ICX(IceLake). 
> > > However,
> > > the upcoming SPR(Sapphire Rapids) has less capabilities of
> > > INTEL_PT_CYCLE_BITMAP than ICX. It fails to enable PT in guest on SPR
> > > machine.
> > > 
> > > If change the check on INTEL_PT_CYCLE_BITMAP to allow different value to
> > > allow it work on SPR. I think it breaks live migration, right?
> > 
> > If I understand your description correctly, I don't think that
> > would break live migration.
> > 
> > Generating different CPUID data for the same CPU model+flags
> > would break live migration.
> > 
> > However, merely allowing a wider set of configurations to work
> > wouldn't break live migration, as long as the same CPU
> > model+flags generates the same CPUID data on any host.
> 
> The easy fix in my brain to make PT work on SPR guest surely will lead to
> different CPUID data between ICX and SPR. That's why I wrote the email.
> 

Yes, but I don't see where the problem is.  We only need to
generate the same CPUID data if you are running the same CPU
model + flags.  It's OK (and expected) to have "-cpu Icelake" and
"-cpu SapphireRapids" result in different CPUID data.


> Other questions about live migration. I think a named CPU model is the
> pre-condition for live migration, right?

Yes.  More precisely, it needs a migration-safe CPU model (in x86
it means all named CPU models except "max" and "host").

> 
> Then is "same QEMU version/binary" the pre-condition for live migration as
> well?

Not at all.  We support migration to different QEMU
binaries/versions.  But "same machine type" and "same -cpu
option" are both preconditions.

> 
> > 
> > > 
> > > For me, not making each sub-function of PT as configurable in QEMU indeed
> > > makes it hard for live migration. [...]
> > 
> > Making all sub-functions of PT configurable would be welcome.
> > actually.  That would allow us to support a wider range of
> > configurations and keep live migration working at the same time.
> 
> I think it will break old QEMU? Is it OK?

If it's a new feature that requires a new command-line option, it
is completely OK.

> 
> > 
> > > [...] Why not make PT as unmigratable in the
> > > first place when introducing the support in QEMU?
> > 
> > I don't know, this was not my decision.  Now we support live
> > migration in a few specific cases (ICX hosts), and I assume we
> > don't want to stop supporting it.
> > 
> > If you just want to support a larger set of hosts when live
> > migration is not needed, you can add support to that use case
> > too.  e.g.: "-cpu host,migratable=off" and/or
> > "-cpu ...,intel-pt-passthrough=on" could do host passthrough of
> > intel-pt sub-leaves (and block live migration).
> > 
> 
> That will make things complicated that old use case "-cpu ...,+intel-pt"
> still means supporting live migration. And when use "-cpu ...,+intel-pt",
> QEMU needs to generate fixed PT capability as previous?

Yes, you need to keep the existing use cases working unless you
deprecate the existing migration-safe use case (which I don't
think you want to).  But a "if (cpu->intel_pt_passthrough)" check
in cpu_x86_cpuid() would solve that.

Anyway, you only need to do that if you want to (if you believe
that's an important and useful use case).

-- 
Eduardo