Re: Rust in QEMU roadmap

[Daniel P . Berrangé]

On Wed, Nov 27, 2024 at 05:47:19AM -0500, Stefan Hajnoczi wrote:
> On Tue, 26 Nov 2024 at 13:04, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > Tracing/logging
> > '''''''''''''''
> >
> > Tracepoints and logging are not supported yet, and no one has started 
> > working
> > on it.
> >
> > For tracing, it's not clear to me how much C code can be reused and how
> > much Rust code can be automatically generated.  This would be a valid
> > Outreachy or Summer of Code project, as it has a limited need for unsafe
> > code and a well-defined scope.
> >
> > A pure-Rust implementation would be interesting, but note that the usage
> > of printf-style %-based field formatting is pervasive in the tracing
> > subsystem.
> 
> The purpose of QEMU's tracing subsystem is to support cross-platform
> tracing. Instead of making QEMU code depend on a specific tracer (like
> DTrace or LTTng UST) that may not be available across platforms, the
> tracing subsystem generates tracer-specific C code for trace events at
> compile time. The most straightforward way to expose this to Rust is
> to add C functions that invoke each trace event and then call those C
> functions from Rust. Tracing is designed to be lightweight and this
> approach results in more overhead than in native C tracing though.
> 
> A pure Rust solution would be nice, but I'm not sure how to reconcile
> that with the purpose of the tracing subsystem, which is to support
> the various tracers native to QEMU's host platforms. They are geared
> towards C and use low-level techniques to generate low overhead
> tracing code.

I think a purer Rust solution looks not too difficult for many of
the backends.

 * dtrace

   Use this crate https://crates.io/crates/usdt to generate
   Rust code from the .d file we already generate

 * ftrace

   A mere write() to a pre-opened FD

 * log

   A call to the qemu_log function, which can be exposed to rust

 * simple

   A call to some funcs in trace/simple.h which can be exposed
   to rust

 * syslog

   A call to syslog(), can be done with https://crates.io/crates/syslog

 * ust

   Unclear how to do this at a quick glance, but likely
   has a solution

Most of these rely on the trace_event_get_state macro which is
where we try to take some fast paths. So the hardest bit is likely
figuring out a nice way to represent the 'state check' in Rust
without it always turning into a function call.

> That's what QEMU's tracing subsystem is today. But it's a low activity
> subsystem where things can be changed with less disruption to users
> than in most other subsystems. I think it would be nice to look at
> what native Rust tracing has to offer ad reconsider some of the
> requirements that shaped QEMU's C tracing subsystem. Maybe we can take
> a more modern approach.

IMHO a core requirement for this must be that we can seemlessly trace
across Rust and C code using the same mechanism.

With my developer hat on, 99% of the time I'm just using the 'log'
backend because it is soooooo simple, very effective for dev use
and we enable it by default.

With my distro hat on, we enable 'dtrace' because that gives us a
low-overhead probe facility for tracing production deployments,
AND crucially it gives us system-global tracing facility across
the kernel, QEMU and all other apps.

IOW, my POV is that support for the "log" and "dtrace" backends
is mandatory for the Rust layer as a minium viable baseline.


I question the usefulness of the 'syslog' trace target. I can't see
it being desirable as a option for 'production' builds, and it seems
uneccessarily indirect for developers. What's its compelling USP ?

WRT 'ftrace', IIUC, the Linux 'perf' command has built-in support
for using USDT probes now:

  https://www.brendangregg.com/perf.html#StaticUserTracing

Do we still have a compelling USP for the 'ftrace' backend, if we
can use 'perf' for this ?

WRT 'ust', in LTTng, I see reference to the ability to use USDT probes,
so I wonder if we still need a dedicated 'ust' backend around ?

Potentially we're down to three options 'log', 'simple', and 'dtrace'.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|