Re: [Qemu-devel] QEMU to generate host binary

[Dr. David Alan Gilbert]

* Ayaz Akram (address@hidden) wrote:
> Thanks for your answers. The thing that i still do not get is once we have
> host assembly code (output assembly log generated for compiled TBs ),
> should we still worry about problems like self modifying code and other
> dynamic conditions? Moreover, assuming static linking, will not this code
> be enough to generate .text section of an executable that could be run
> directly on host (if somehow other sections of that host executable can be
> generated, which is itself difficult) ?

It can certainly help, but you can still walk into a piece of code
that you've not previously translated, and a shared library for example
might change, and your code might change due to other things such as
randomisation of library loading addresses.

Various emulators in the past have dumped some part of the translation state
to disk in an attempt to save on costs when they next emulate the code;
others have tried to do background analysis to optimise the translated
code into a pretranslated blob - but they can never guarantee to get
the whole set of code translated; and that's even for a user-mode emulation
where at least you know which binary file you're trying to emulate.

In system emulation it's much more difficult because the emulator doesn't
have a view of which executable the page it's currently emulating corresponds
to; indeed that page could be shared between multiple different executables,
or in the opposite direction is often patched during runtime linking.

You say 'should we still worry about problems like self modifying code and other
dynamic conditions?' - well you hope that for most 'normal' programs that
self modification doesn't happen; but it keeps turning up even when you don't
expect it, e.g. loading/unloading of plugins or if you happen to be
emulating a JIT (such as qemu!).  And QEMU (especially in system mode)
doesn't know whether the binary that's being run is a nice safe friendly
static binary or something that's going to do some mad optimisation itself.

One example of the type of thing you can do is save statistics information
about a piece of executable, e.g. 'this code is hot' so you know it's
a hot loop that's worth optimising; that can pay off if you have multiple
levels of interpreter/optimisation in your emulator.

Dave

> 
> 
> 
> On Mon, Jun 29, 2015 at 1:04 PM, Peter Crosthwaite <
> address@hidden> wrote:
> 
> > On Mon, Jun 29, 2015 at 8:13 AM, Stefan Hajnoczi <address@hidden>
> > wrote:
> > > On Sun, Jun 28, 2015 at 07:29:39PM -0400, Ayaz Akram wrote:
> > >> > Let's say qemu is running in System Emulation Mode, when it runs
> > guest's
> >
> > System emulation makes the problem even harder, as a system mode
> > binary (usually an OS or some sort) will have difficult porting from
> > one CPU-types system arch to another.
> >
> > This is more realistic (but still very difficult and not generally
> > solvable) in user-mode emulation.
> >
> > >> > binary, it can log the translated code for host. Is it possible to
> > merge
> > >> > that translated code and other sections of guest's binary to make a
> > binary
> > >> > which can be run directly on host.
> > >
> > > No, because of self-modifying code, run-time code loading, etc.
> > >
> >
> > Ruling these two out for the moment ...
> >
> > > It is not possible to statically translate an executable (in the general
> > > case).
> > >
> > > There are architectures where it is possible due to restrictions (e.g.
> > > no code loading, all jump destinations are known in advance, etc) but
> >
> > Debug info with function information might give you a crude
> > approximation of jump targets coming from fn pointers. That + the
> > statically determinable jump targets might give you something for apps
> > that don't do anything wierd.
> >
> > I'm wondering if the jump problem can be crudely solved by a fully
> > single-step translation. The result binary would be huge an
> > inefficient. But could you keep two translations around? One that uses
> > the statically determinable "best guess" of the jump dest table I
> > describe above, and a second defensive translation of the entire app
> > in single-step?
> >
> > There are more complications however. Another one I can think of is
> > instructions that change runtime state and affect (re)translation
> > (e.g. the arm setend instruction which switches CPU endianness).
> >
> > Regards,
> > Peter
> >
> > > the popular x86, ARM, etc architectures allow too much freedom to be
> > > amenable to static translation.
> > >
> > > Stefan
> >
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK