Re: [PATCH v5 1/5] iotests: remove 'linux' from default supported platforms

[Peter Maydell]

On Mon, 7 Oct 2019 at 13:32, Thomas Huth <address@hidden> wrote:
> On 04/10/2019 14.44, Max Reitz wrote:
> > Whenever make check fails, it’s urgent.  Without iotests running in make
> > check, we had some time to sort the situation out.  That’s no longer the
> > case.
> >
> > That means we need to take care of everything immediately.  And I purely
> > want help with that.
>
> While that sounds noble at a first glance, I think you've maybe got too
> high expectations at yourself here? I mean, all other maintainers of
> other subsystems with tests have the same "problem" if the tests for
> their subsystem run in "make check". The "normal" behavior that I've
> seen so far (and which I think is the right way to deal with it):
>
> 1) If a pull request gets rejected due to a "make check" failure, simply
> drop the offending patches from the pull request, send a v2 pull req
> without them, and tell the author of the offending patches to fix the
> problem (unless the fix is completely trivial and could immediately be
> applied to the v2 pull req). You really don't have to fix each and every
> issue on your own as a maintainer and can redirect this to the patch
> authors again.
>
> 2) If a test fails occasionally during "make check" (due to race
> conditions or whatever), it gets disabled from "make check" if it can't
> be fixed easily (e.g. in the qtests it gets moved to the "SPEED=slow"
> category, or in iotests it would get removed from the "auto" group).

I would agree with this and would also suggest that things tested
in 'make check' should ideally be reducing work for the maintainer:
if something fails 'make check' then that change won't get into
the tree and the task of getting it to work is pushed back to the
original patch submitter. If something causes failures but they're
not caught by 'make check' then the patch will get into the tree and
now it's likely the subsystem maintainer's job to track down and
fix the bug after the fact.

> So if you like, I can send a patch to remove 130 from the "auto" group
> (personally, I'd rather wait to see if it fails anytime soon again, or
> if this was maybe rather a one-time issue due to a non-clean test system
> ...)

FWIW I haven't seen that iotest 130 failure again...

thanks
-- PMM