Re: [BUG] Migration hv_time rollback

[Antoine Damhet]

On Wed, Sep 16, 2020 at 01:59:43PM +0200, Vitaly Kuznetsov wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> 
> > cc'ing in Vitaly who knows about the hv stuff.
> >
> 
> cc'ing Marcelo who knows about clocksources :-)
> 
> > * Antoine Damhet (antoine.damhet@blade-group.com) wrote:
> >> Hi,
> >> 
> >> We are experiencing timestamp rollbacks during live-migration of
> >> Windows 10 guests
> 
> Are you migrating to the same hardware (with the same TSC frequency)? Is
> TSC used as the clocksource on the host?

Yes we are migrating to the exact same hardware. And yes TSC is used as
a clocksource in the host (but the bug is still happening with `hpet` as
a clocksource).

> 
> >>  with the following qemu configuration (linux 5.4.46
> >> and qemu master):
> >> ```
> >> $ qemu-system-x86_64 -enable-kvm -cpu host,kvm=off,hv_time [...]
> >> ```
> 
> Out of pure curiosity, what's the purpose of doing 'kvm=off'? Windows is
> not going to check for KVM identification anyway so we pretend we're
> Hyper-V. 

Some softwares explicitly checks for the presence of KVM and then crash
if they find it in CPUID :/

> 
> Also, have you tried adding more Hyper-V enlightenments? 

Yes, I published a stripped-down command-line for a minimal reproducer
but even `hv-frequencies` and `hv-reenlightenment` don't help.

> 
> >
> > How big a jump are you seeing, and how did you notice it in the guest?
> >
> > Dave
> >
> >> I have tracked the bug to the fact that `kvmclock` is not exposed and
> >> disabled from qemu PoV but is in fact used by `hv-time` (in KVM).
> >> 
> >> I think we should enable the `kvmclock` (qemu device) if `hv-time` is
> >> present and add Hyper-V support for the `kvmclock_current_nsec`
> >> function.
> 
> AFAICT kvmclock_current_nsec() checks whether kvmclock was enabled by
> the guest:
> 
>    if (!(env->system_time_msr & 1ULL)) {
>         /* KVM clock not active */
>         return 0;
>     }
> 
> and this is (and way) always false for Windows guests.

Hooo, I missed this piece. When is `clock_is_reliable` expected to be
false ? Because if it is I still think we should be able to query at
least `HV_X64_MSR_REFERENCE_TSC`

> 
> >> 
> >> I'm asking for advice because I am unsure this is the _right_ approach
> >> and how to keep migration compatibility between qemu versions.
> >> 
> >> Thank you all,
> >> 
> >> -- 
> >> Antoine 'xdbob' Damhet
> 
> -- 
> Vitaly
> 

-- 
Antoine 'xdbob' Damhet

signature.asc
Description: PGP signature