Re: Advancing to Patch::review

[James]

On 03/12/2020 11:46, Michael Käppler wrote:
> What do you mean with "full reg test suite"? Additional examples that
> are not in the normal reg test suite?

Well in the 'olden days' we'd have a set of tests on the website - here 
is an old URL that no longer works 
(http://lilypond.org/test/v2.13.51-1/compare-v2.13.50-1/index.html 
<http://lilypond.org/test/v2.13.51-1/compare-v2.13.50-1/index.html>) - 
and as this was autogenerated as part of the website build, it was quick 
and easy to see if there were any changes that had altered between, say, 
last unstable or between current unstable and previous stable.

> I think comparisons between several versions are of limited usefulness,
> because it can be difficult to see if
> changes are intended, not intended, or even occuring at random.

Quite, but if you do this 'check' as part of the workflow for the next 
official release (Phil may already, I don't know and I don't want to 
necessarily make more work for anyone) then at least we aren't in for 
any shocks and also, anything that is intended should be in the 
changes.tely file, so this also helps us keep that up to date. But I am 
not sure how complex this is to set up again and if it is worth the work 
or not.

> In my opinion the current test suite has conceptual problems that go
> beyond that discussion.
> E.g. that we don't compare actual output, but grob positions and
> extents, AFAIK.
> That lead to problems in Harm's MR:
> https://gitlab.com/lilypond/lilypond/-/merge_requests/497
> Also issues like https://gitlab.com/lilypond/lilypond/-/issues/720 that
> affect specific backends cannot be
> tested (or it would require hacks)

Yes that is beyond the scope of what I am talking about, but 
fundamentally it might be useful to someone ('perfect enemy of good' and 
all that).


James