Re: [PATCH] scripts: display how long each test takes to execute

[Paolo Bonzini]

On 14/09/20 13:33, Daniel P. Berrangé wrote:
> On Mon, Sep 14, 2020 at 01:19:20PM +0200, Paolo Bonzini wrote:
>> On 14/09/20 13:09, Daniel P. Berrangé wrote:
>>> I'm unclear if meson's native test runner can print timings. If not,
>>> we might want to submit an RFE there too.
>>
>> I agree that any holes should be filled in there.  In this case it does,
>> so I think we should start using it in CI so that RFEs can be sent there.
>>
>> mtest2make-style output has been in use (and good enough) for years so
>> I'd rather avoid piling up more hacks on top.  "meson test" is not
>> perfect but I'd rather improve it instead.
> 
> When I run "meson test" it doesn't use the results from the previous
> "make", instead it re-compiles the entire codebase using ninja.

Yes, you'd have to add --no-rebuild.

> If we're telling users to continue to use "make" and "make check" though,
> I don't think we should be using "meson test" in the CI systems, as it
> means CI is not testing the same build process as our users, which is
> defeating the point of CI.

We can of course cover both.

> If using "meson compile" and "meson test" already works though, what
> is our current ninja -> make convertor doing for us, besides letting
> people have a facade to pretend nothing has changed ?

Before unit tests were converted, it was needed because unit tests were
built part with Meson and part with Makefile rules.  Now we still need
the Makefile to build submodules.  Richard posted a patch to build
capstone from meson.build; libfdt can have the same treatment and SLIRP
can be converted to a subproject.  With that gone, there's no reason to
keep the ninja2make converter.

Even before that, it's also possible to send the build to Ninja instead
of using ninja2make.  In that case the Makefile would only spawn
recursive builds using either Make or Ninja.  That would also add a
Ninja build dependency, so I haven't proposed it yet (despite having the
patches ready).

To sum up: the hard part is done, but that means that we have to decide
in what order to do the next steps.

Paolo "what are the Romans doing for us?"