Re: issue verification

[Jean Abou Samra]

Hi,

Le 18/09/2020 à 11:15, Phil Holmes a écrit :

> I don't know if this isn't clear, but just to state the original point 
> of verifying issues.
>
> If the change is claimed to fix a bug, we compile the previously buggy 
> code with the release in which the bug is claimed fixed and check that 
> the bug is no longer there.  It it has really disappeared, we change 
> the status of the issue from Fixed to Verified.  i.e. we are certain 
> that the bug is no longer there.
>
> If the change is to provide updated functionality, then it can be 
> really quite hard to verify the new functionality, and in any case the 
> patch review system should do that.  So we simply check the patch was 
> pushed into the claimed build.  If it's clear that it was, we mark the 
> status as Verified.
>
> That was the original intention of verifying issues.

Thanks for the insight.

So, Michael and I verified nearly all issues in Status::Fixed.
The ones remaining are listed here:

https://gitlab.com/lilypond/lilypond/-/issues?scope=all&utf8=%E2%9C%93&state=closed&label_name[]=Status%3A%3AFixed&not[milestone_title]=2.21.7

Among these,

https://gitlab.com/lilypond/lilypond/-/issues/6027

which I don't know how to test (Jonas?).

The others (5 items) are Windows-related. Maybe Michael or someone
else with access to a Windows machine can go over them?

Otherwise, part of the work was extremely simple, namely check that
the commits had made it into some branch under release/, which is fast;
and part of it was much harder, when trying to reproduce memory-related
bugs, for example. I reopened a few of these issues.

This doesn't answer Jonas' initial question: what should be done,
in terms of policies, with regard to issue verification? I don't
know. Personally I think it's not really needed as long as every
bug fix contains a regression test.

Speaking of regression tests, I also went through comparison pages
from "2.21.6 vs. 2.21.5" to "2.21.2 vs. 2.21.1" included. Honestly,
I've had my share of this; could somebody jump in to verify the
last two pages as well? These are:

lilypond.org/test/v2.21.1-1/compare-v2.21.0-1/index.html

http://lilypond.org/test/v2.21.0-1/compare-v2.20.0-1/index.html

Depending on your time and energy available, you may even go
farther. Or simply (I mean simple, not costless) verify the
entire test suite:

http://lilypond.org/doc/v2.21/input/regression/collated-files.html

That'll be all for today.

Best,
Jean