Re: [Qemu-devel] [Xen-devel] [RFC PATCH 0/2] support xen HVM direct kernel boot

[Chun Yan Liu]

>>> On 6/3/2014 at 05:31 PM, in message
<address@hidden>, Ian Campbell
<address@hidden> wrote: 
> On Tue, 2014-06-03 at 03:16 -0600, Chun Yan Liu wrote: 
> >  
> > >>> On 6/2/2014 at 11:14 PM, in message 
> > <address@hidden>, Stefano 
> > Stabellini <address@hidden> wrote:  
> > > On Fri, 30 May 2014, Konrad Rzeszutek Wilk wrote:  
> > > > On Thu, May 29, 2014 at 11:23:22AM +0800, Chunyan Liu wrote:  
> > > > > Following previous discussion:  
> > > > > https://lists.gnu.org/archive/html/qemu-devel/2014-04/msg03843.html  
> > > > >   
> > > > > I spent lot of time to understand xen xc_hvm_build, hvmloader code,  
> > > > > and qemu pc_memory_init, xen_ram_init code, and wrote experimental  
> > > > > code to test. Now it's basically working based on seabios.  
> > > >   
> > > > Excellent!  
> > > > >   
> > > > > Talking about qemu linuxboot.bin/multiboot.bin first, they are  
> > > > > implemented to intercept boot process, originally by override int19,  
> > > > > later changed to provide BEV entry which will be called by seabios  
> > > > > if option rom is selected for booting. linuxboot.bin/multiboot.bin  
> > > > > will load kernel/initrd and jump to execute kernel directly.  
> > > > >   
> > > > > In qemu load_linux() code, it first read out kernel/initrd file and  
> > > > > store related ADDRs to fw_cfg (from where linuxboot.bin/multiboot.bin 
> > > > >  
> > > > > will get the ADDRs and load kernel/initrd data), then add  
> > > > > linuxboot.bin/multiboot.bin to option rom with bootindex=0.  
> > > > >   
> > > > > In seabios side, it could load and execute ROMs by itself, and 
> > > > > according  
>  
> > > > > to boot order, it will boot from the BEV entry on the option rom. 
> > > > > That  
> > > > > will then be taken over by linuxboot.bin/multiboot.bin.  
> > > > >   
> > > > > So, in theory, qemu load_linux() code almost does the work. I just 
> > > > > tried  
>  
> > > > > to reuse it for xen.  
> > > > >   
> > > > > xen side patch: pass kernel/initrd/append parameters to qemu-dm  
> > > > > qemu side patch: reuse load_linux() for xen hvm direct kernel boot.  
> > > > >     Different from pc_memory_init which does lots of ram alloc work  
> > > > >     and rom/bios loading work, for xen, we only need to init a basic  
> > > > >     fw_cfg device used by load_linux() to store ADDRs and  
> > > > >     linuxboot.bin/multiboot.bin to retrive ADDRs, then load_linux(),  
> > > > >     after that, do real add option rom work to add  
> > > > >     linuxboot.bin/multiboot.bin to system option rom. Other things  
> > > > >     would be done by seabios smoothly.  
> > > > >   
> > > > > In my testing, it's working based on seabios.  
> > > > > Rombios case is not working. But now that xen prefers qemu upstream,  
> > > > > seabios will be default, do we need to put effort to make rombios 
> > > > > work?  
> > > >   
> > > > No. But the code (libxl) should detect whether you are using the  
> > > > traditional QEMU - which would be using ROM BIOS. If it detects that  
> > > > it shouldn't allow you to pass the arguments.  
> > > >   
> > > > You probably will also need to an docs/ change to document these  
> > > > new parameters.  
> > > > > And stubdom case, since I didn't make my guest started even without  
> direct  
> > > > > kernel boot, I don't know if it works. And I still could not well  
> > > > > understand about the differences to stubdom and non stubdom this work 
> > > > >  
> > > > > may cover. Any problems please point out.  
> > > >   
> > > > Hmm, you would have to somehow 'transfer' the images from one domain  
> > > > (the control domain) to the stubdomain which would do the execution.  
> > > >   
> > > > At least the multiboot.bin image.  
> > >   
> > > Yeah, unfortunately the approach of implemeting direct kernel boot for  
> > > HVM guests via QEMU wouldn't work for stubdomains.  
> > >   
> > > If we want this to work with stubdoms, it would be best to reimplement  
> > > kernel and initrd loading in libxl/libxc.  
> >  
> > Thanks. Rouphly thinking, the implementation might be totally different. 
> > For seabios, could load kernel/initrd (and linuxboot.bin) in libxc, and in 
> > qemu find a way to get data and continue with existing code logic (qemu 
> > side changes may be not very clean). 
>  
> I suppose the native qemu code path is roughly 
>       * qemu: 
>               * parse command line 
>               * load file from disk into appropriate place in guest RAM 
>               * insert the BIOS extension 
>               * go 
>  
> For Xen it would perhaps be something like: 
>       * libxc: 
>               * load file from disk into appropriate place in guest RAM 
>               * communicate this to qemu somehow (xenstore, command 
>                 line, in memory 
>               * data structure at known location, carrier pigeons) 
>       * qemu: 
>               * discover images in RAM according to scheme decided above 
>               * (perhaps) move those images to the appropriate place in 
>                 guest ram 
>               * insert the BIOS extension 
>               * go 

Thanks very much! That's what I want to do.
The main concern is the linuxboot.bin/multiboot.bin image. We should
also load linuxboot.bin/multiboot.bin in libxc first. In native qemu, it inserts
option rom through rom_add_option("linuxboot.bin", 0). Now since
linuxboot.bin has been loaded to RAM, we could not use existing
rom_add_option or other existing APIs, possibly need to write own
rom_add function.

Another concern is, linuxboot.bin/multiboot.bin are binaries generated
by qemu, so there should be a fixed palce for xen to find
linuxboot.bin/multiboot.bin. If stubdom supports qemu upstream, it should
export linuxboot.bin/multiboot.bin to somewhere after xen installation.

- Chunyan

>  
> So at least the second half of the qemu side could be common/normal. 
> Perhaps the first half on the qemu side could be abstracted away as some 
> sort of magic "file like" thing backed by bytes already present in guest 
> RAM? (it should be pretty clear by now that I don't really know much 
> about the qemu internals side of this...!) 
>  
> >  And stubdom-dm uses 
> > qemu-xen-traditional, that means rombios by default, that method doesn't 
> > work. 
>  
> Sorry, I should have said: support for direct kernel from a stub qemu is 
> dependent on someone implementing stubdm with qemu upstream first. There 
> is no need at all to implement this for the existing qemu-trad stub dm, 
> in fact I'm actively against doing so (no new features for qemu-trad). 
>  
> Ian Jackson is working on making qemu upstream work as a stubdom. 
>  
> I think I'd be happy if we simply had a plausible plan for how we would 
> support direct kernel boot with upstream qemu in a stubdm, rather than 
> blocking these changes on that or expecting you to do that porting work. 
> Ian -- does that sound OK? 
>  
> >  Maybe need to change hvmloader? I'll have a look. 
> >  
> > > I have to admit that given the extreme simplicity of this patch series,  
> > > I wouldn't be against merging it, even if it cannot be made to work with  
> > > stubdoms.  However like in the rombios case, libxl would need to  
> > > understand that this feature doesn't work with stubdoms and print an  
> > > appropriate message.  
> >  
> > I'll update printing messages. But now, as you prefer, should I update 
> > existing patch series to let it be in good shape for merge, or should I 
> > abandon it and reimplement to try to make it work with stubdom? 
>  
> You can update this series, I think. 
>  
> Ian. 
>  
>  
>