[Qemu-devel] Xen hypervisor inside KVM guest with x2apic CPU feature fails to boot

[Daniel P. Berrange]

I'm running

 kernel-3.14.4-200.fc20.x86_64
 qemu-1.6.2-5.fc20.x86_64
 xen-4.4.0-4.fc21

In process of trying to get a Xen hypervisor running inside a KVM guest I
found that there's a problem with x2apic. NB I do *not* use nested-VMX
here, just trying to get plain Xen paravirt working before trying todo
nested HVM.

Any time I enable the 'x2apic' CPU flag for the KVM guest, the Xen hypervisor
running inside the guest will fail to boot:

The QEMU/KVM -cpu arg is

  -cpu 
core2duo,+erms,+smep,+fsgsbase,+lahf_lm,+rdtscp,+rdrand,+f16c,+avx,+osxsave,+xsave,+aes,+tsc-deadline,+popcnt,+x2apic,+pcid,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds
 

The Xen logs indicate it isn't liking the x2apic feature and is disabling
it, but then it obviously fails to setup the non-x2apic codepath it is
following - even though the non-x2apic codepath works fine if you don't
have +x2apic set for the KVM guest.

(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) XSM Framework v1.0.0 initialized
(XEN) Flask:  Initializing.
(XEN) AVC INITIALIZED
(XEN) Flask:  Starting in permissive mode.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2693.939 MHz processor.
(XEN) Initing memory sharing.
(XEN) traps.c:3071: GPF (0000): ffff82d0801b83c7 -> ffff82d08023386b
(XEN) mce_intel.c:717: MCA Capability: BCAST 1 SER 1 CMCI 0 firstbank 1 
extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) I/O virtualisation disabled
(XEN) Getting VERSION: 1050014
(XEN) Getting VERSION: 1050014
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Getting ID: 0
(XEN) Getting LVT0: 8700
(XEN) Getting LVT1: 8400
(XEN) Suppress EOI broadcast on CPU#0
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) init IO_APIC IRQs
(XEN)  IO-APIC (apicid-pin) 0-0, 0-16, 0-17, 0-18, 0-19, 0-20, 0-21, 0-22, 0-23 
not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
(XEN) ...trying to set up timer (IRQ0) through the 8259A ...  failed.
(XEN) ...trying to set up timer as Virtual Wire IRQ... failed.
(XEN) ...trying to set up timer as ExtINT IRQ... failed :(.
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) IO-APIC + timer doesn't work!  Boot with apic_verbosity=debug and send a 
report.  Then try booting with the 'noapic' option
(XEN) ****************************************

Will attach the full non-trimmed Xen log to this mail, along with a log
showing successful boot when 'x2apic' isn't given to KVM.

I'm unclear if this is a Xen bug or KVM bug or QEMU bug, or a combination
of them....

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

xen-x2apic.log
Description: Text document

xen-normal.log
Description: Text document