Re: [Qemu-devel] [PATCH v2 4/4] hw/i386: Introduce the microvm machine type

[Sergio Lopez]

Paolo Bonzini <address@hidden> writes:

> On 18/07/19 16:34, Sergio Lopez wrote:
>> I've just added support for starting the machine from SeaBIOS (Stefan
>> Hajnoczi pointed in another thread that it can be as fast as qboot, and
>> given that the latter doesn't support mptables, I just tested this
>> one). I tried to keep it as minimalistic as possible, but it still
>> required an RTC (mc146818), which dragged in an ISA BUS, and this one a
>> KVM PIT.
>> 
>> I ran some numbers using Stefano Garzarella's qemu-boot-time scripts
>> [1] on a server with 2xIntel Xeon Silver 4114 2.20GHz, using the
>> upstream QEMU (474f3938d79ab36b9231c9ad3b5a9314c2aeacde) built with
>> minimal features [2]. The VM boots a minimal kernel [3] without initrd,
>> using a kata container image as root via virtio-blk (though this isn't
>> really relevant, as we're just taking measurements until the kernel is
>> about to exec init).
>> 
>>  ---------------------
>>  | QEMU with SeaBIOS |
>>  ---------------------
>> 
>>  qemu_init_end: 65.958714
>>  linux_start_kernel: 77.735803 (+11.777089)
>>  linux_start_user: 127.360739 (+49.624936)
>> 
>>  -------------------
>>  | QEMU direct PVH |
>>  -------------------
>> 
>>  qemu_init_end: 64.043264
>>  linux_start_kernel: 65.481782 (+1.438518)
>>  linux_start_user: 114.938353 (+49.456571)
>> 
>>  --------------
>>  | Comparison |
>>  --------------
>> 
>> Average boot time:
>> 
>>  * Relying on SeaBIOS, when compared with direct PVH boot, as a total
>>    average overhead of ~12ms. The cost of initializing QEMU increases in
>>    ~2ms (probably due to need to instantiate more devices), while the
>>    other ~10ms is the SeaBIOS overhead.
>> 
>>  ---------------
>>  | Conclusions |
>>  ---------------
>> 
>> Objectively, the version that boots directly the kernel using PVH is 10%
>> faster and has a slightly larger exposed surface. Whether this is enough
>> to justify its existence is quite subjective.
>> 
>> In my opinion, not only I think it makes sense to have it, but I also
>> think there's little reason to have the firmware reliant version, given
>> the nature and purpose of microvm.
>
> The advantage of firmware is support for vmlinuz and multiboot in
> addition to PVH, and removing code from QEMU.  So I think it's still
> worth doing a comparison with qboot, and trying to understand where
> SeaBIOS is spending its time (qboot should not need additional devices
> other than fw_cfg, and since SeaBIOS has never been optimized for
> microvm I expect it's possible to shave quite a few of those 12 ms).

I tweaked qboot a bit to make it work with microvm (basically, if it
can't identify a PCI vendor, it assumes it's running in a microvm, and
avoids PCI and ACPI initialization, writing an minimal mptable instead)
and I've just got some numbers:

Average boot time:
 qemu_init_end: 64.502199
 linux_start_kernel: 66.644311 (+2.142112)
 linux_start_user: 116.279127 (+49.634816)

To avoid having to scroll up, these are the numbers with the direct PVH
boot version:

Average boot time:
 qemu_init_end: 64.043264
 linux_start_kernel: 65.481782 (+1.438518)
 linux_start_user: 114.938353 (+49.456571)

So starting from qboot is just a bit over 1ms slower, which I think it's
quite reasonable. And, as you said, the only additional device is
fw_cfg:

address-space: I/O
  0000000000000000-000000000000ffff (prio 0, i/o): io
    000000000000007e-000000000000007f (prio 0, i/o): kvmvapic
    0000000000000510-0000000000000511 (prio 0, i/o): fwcfg
    0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma

If people feel more comfortable keeping QEMU on x86_64 starting on Real
Mode and having the firmware writing mptables, I'm fine with taking this
approach.

That said, to avoid confusing users, I think it'd be better to drop the
legacy mode for SeaBIOS, and support just qboot. That would mean having
to add a binary from the latter to "pc-bios" with some descriptive name
("qboot-microvm.bin"?) and making microvm look for that one by default.

Does this sound like a reasonable plan?

> On the other hand I agree that microvm is showing great promise compared
> to PCI-based machine types.
>
> Paolo

signature.asc
Description: PGP signature