Re: [PATCH v5 1/5] iotests: remove 'linux' from default supported platforms

[Thomas Huth]

On 02/10/2019 13.57, Max Reitz wrote:
> On 02.10.19 06:46, Thomas Huth wrote:
>> [...]
>> Max, I can understand that you are a little bit annoyed that this "make
>> check with iotests" caused some extra hurdles for you. But honestly,
>> removing that again would be quite egoistic by you. Try to put yourself
>> into the position of a "normal", non-blocklayer-maintainer QEMU
>> developer. For me, iotests were a *constant* source of frustration.
>> Often, when I ran them on top of my latest and greatest patches, to
>> check whether I caused a regression, the iotests simply failed. Then I
>> had to start debugging - did my patches cause the break, or is "master"
>> broken, too? In almost all cases, there was an issue in the master
>> branch already, either because they were failing on s390x, or because I
>> was using ext4 instead of xfs, or because I was using an older distro
>> than you, etc... . So while the iotests likely worked fine for the
>> limited set of systems that you, Kevin and the other hard-core block
>> layer developers are using, it's constantly broken for everybody else
>> who is not using the very same setup as you. The only way of *not*
>> getting upset about the iotests was to simply completely *ignore* them.
>> Is it that what you want?
> 
> It usually worked fine for me because it’s rather rare that non-block
> patches broke the iotests.
> 
> I have to admit I actually didn’t think of other people wanting to run
> the iotests; but to be honest, your mail doesn’t sound like you want to
> run the iotests either.

I *want* to run them. Occasionally - when I have new patches that might
affect anything related to the block layer. But then I don't want to be
in the situation where I first have to debug multiple other problems
with the iotests first that are not related to my new patches.

> (The reason I didn’t think of it is because non-block patches rarely
> break them, so I wouldn’t run the iotests if I were a non-block
> maintainer.  Sorry.)

Well, "rarely" is relative. They've been broken *completely* two times
in the 4.1 development time frame, and IIRC at least once in the 4.0
time frame.

[...]
> Maybe my main problem is that I feel like now I have to deal with all of
> the fallout, even though adding the iotests to make check wasn’t my idea
> and neither me nor Kevin ever consented.  (I don’t know whether Kevin
> consented implicitly, but I don’t see his name in bdd95e47844f2d8b.)
> 
> You can’t just give me more responsibility without my consent and then
> call me egoistic for not liking it.

Ok, sorry for that. I guess one part of my frustration was also that the
patches to enable the iotests during "make check" have been on the list
for weeks - or rather months - and I never ever got much feedback from
you or Kevin on them. If you told me right in the beginning about your
concerns, we would not be at this point now. Also partly my bad, I
guess, since I could have reached out to you on IRC to discuss it, but
at that point in time, I rather thought that you just don't really care.
Thus it's good to have some conversation now, helps a lot to understand
the different expectations. Maybe we can also have a discussion about
this at KVM forum, I think it's easier to clarify some points of view
verbally there instead of using mails.

>> Or maybe let me phrase it differently: Do you consider the iotests as
>> something that is more or less "private" to the hard-core block layer
>> developers, and it's ok if others completely ignore them and break them
>> by accident (and you also don't expect the normal developers to fix the
>> iotests afterwards)?
> 
> Well, that’s how it’s always worked, and that didn’t frustrate me.

Ok ... you're the maintainer, so if that's really the way you prefer, I
can send a patch to remove the iotests from "make check" again.

> Honestly, it looks to me like you don’t even want to run the iotests.  I
> interpret most of what you’ve written as:
> 
> - I don’t want to not run the iotests, because then people will hit me
>   for making them fail.
> 
> - But they fail all the time, so I always need a baseline for what is
>   expected to sometimes fail and what isn’t.  That’s very annoying.
>   Let’s introduce a baseline in the form of auto/qcow2, and then let
>   everyone verify that it works.
> 
> So to me it looks like we’ve just added all tests that never fail to
> auto.  But if they never fail, then it’s like we haven’t run the tests
> at all.

I disagree. First, the complete iotest failures that have been merged
during the 4.1 development time frame to the master branch would have
been prevented, I think, so it's certainly not that "they never fail".

Second, yes, the basic idea was to start with a small set of tests in
the auto group which was already working, and then to increase that set
over time, once other tests run more stable, too. But you know the
iotests better than me, if you think that most of them can hardly
brought into the right shape, then this was likely just wishful thinking
by me.

> You have to decide: Either let me deal with the problems, but then I
> have every right to be egoistic about it – or you help me deal with them.

Since I'm not assigned to the block layer, I could only help
occasionally, so that's likely not enough for solving your frustration.
Thus I'll send a patch to remove the iotests from "make check" again.

 Thomas