Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Christoffer Dall]

Hi,

[sorry for reviving this thread late]

On Tue, Jun 09, 2015 at 12:24:13PM +0100, Peter Maydell wrote:
> On 9 June 2015 at 11:52, Marc Zyngier <address@hidden> wrote:
> > On 08/06/15 11:52, Peter Maydell wrote:
> >> On 8 June 2015 at 11:32, Igor Mammedov <address@hidden> wrote:
> >>> On Thu, 4 Jun 2015 18:17:39 +0100
> >>> Peter Maydell <address@hidden> wrote:
> >>>> On 4 June 2015 at 17:40, Shlomo Pongratz <address@hidden> wrote:
> >>>>  In order for it to work correctly we must use MPIDR values in
> >>>>  the device tree which match the MPIDR values the kernel has picked
> >>>>  for the vcpus, so ask KVM what those values are.
> >>
> >>> Could we set QEMU's generated mpidr in kernel instead of pulling it from 
> >>> kernel,
> >>> like we do with APIC ID in x86 and fix kernel not to reset it its own 
> >>> value
> >>> (i.e. untie mpidr from vcpuid)?
> >>>
> >>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
> >>> board code which should be setting it, since it knows/defines what 
> >>> topology it has.
> >>
> >> This is a question better asked on the kvmarm list (which I have cc'd),
> >> because that is where the kernel folks hang out...
> >
> > Care to provide some context? Why is this required?
> 
> The device tree spec says we have to tell the guest kernel the
> affinity information for each CPU, in the reg field of the CPU node.
> This device tree blob is created by userspace; the MPIDR the guest
> actually reads is currently created by the kernel. The kernel's
> vgic v3 code may also make assumptions about vcpu_id => affinity
> mappings, I'm not sure.
> 
> In current code:
>  * the kernel has an opinion about vcpu_id => MPIDR mappings
>  * QEMU does too, but it's not the same!
>    (luckily for 8 or fewer CPUs they agree, but this will not
>    do for GICv3 support)
> 
> So either:
>  * QEMU needs to tell the kernel the MPIDR for each vCPU

Does this option work for all configurations?  What if you're running on
a hardware with two clusters [8, 8] and QEMU asks KVM for a [12, 2]
configuration, for example.  Will it not be impossible for KVM to
maintain correct semantics for coherency guarantees across what seems to
the guest to be the same cluster, but really is two different clusters?

Of course, KVM can deny an unsupported configuration, but I am wondering
if we really think anybody will care about the 'model such specific
hardware' aspect with KVM, or if we should only consider the 'I want a
VM with x VCPUs' scenario, in which case the second option below seems
simpler to me.

Are there any concerns to think about wrt. migration here as well?

Thanks,
-Christoffer

>  * QEMU needs to ask the kernel the MPIDR for each vCPU
> 
> Which is better? The latter is simpler and will work with
> existing kernels. The former would let us (for instance)
> use KVM when we're modelling (real world) boards which have
> particular cluster configurations (which might not match the
> kernel's current simplistic "always 16 CPUs at Aff0" setup).
> 
> thanks
> -- PMM
> _______________________________________________
> kvmarm mailing list
> address@hidden
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm