Re: Reg test diffs for XML - noise in all tests

[Han-Wen Nienhuys]

On Sun, Jul 19, 2020 at 1:16 PM Jonas Hahnfeld <hahnjo@hahnjo.de> wrote:
> > > thanks for raising this. Since MR !230, 'make check' started comparing
> > > the regression tests for musicxml and there should be image diffs in
> > > case of changes (see the long output of 'make check' for !230 which
> > > also contained the fix for chords).
> >
> > I actually think this is not the way to go for XML. Rather than
> > compare PNG outputs (which mainly measures changes in the formatting
> > engine), we should compare the XML files against baseline as text
> > files.
>
> I think that's a misunderstanding: The tests take MusicXML, run
> musicxml2ly to produce an input for LilyPond and compare the result. We
> could of course compare the (textual) input to LilyPond, but I'd argue
> that comparing visual representations is easier for humans.

I had to look into an XML regtest problem yesterday, and resorting to
textual comparison of the .ly was actually more productive.

> > > Moreover, cell counts won't be available with Guile v2.x (at least the
> > > property 'total-cells-allocated doesn't exist anymore). Would anybody
> > > object to dropping this for Guile v1.8 already? That would potentially
> > > also simplify the introduction of automated regression test comparison
> > > in the CI pipeline...
> >
> > I'm also in favor of dropping this.
>
> Good, I'll prepare the changes. Can I just drop the options dump-
> profile and dump-cpu-profile or are there external users?

no, go ahead.
-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen