Re: GSOC Build tests

[Werner LEMBERG]

> I've finished the core of the regression tester.

Great!

> You can now run it and generate a html report but you will need a
> few tools installed: imagick, xwd, npm, pretty-diff and xvfb.  This
> should all be listed in ft-regression.sh's comments.If you want to
> run it locally make sure you have a ~/test-fonts with .ttf files in
> it.

Thanks; I will try that in the next few days, hopefully.

> This is basically what I wanted to accomplish for my GSOC project
> but I can do a lot of the things mentioned in my previous message if
> I pass my evaluation.

:-)

> You can download the report it generates from here:
> https://dev.azure.com/fundies/9eabb07a-6a4d-4b68-b22e-60f9e02c1927/_apis/build/builds/232/artifacts?artifactName=Archlinux%20Regression%20tests&api-version=5.1&%24format=zip
> The file is rather large.  In the future, I could probably shrink it
> quite a bit by using 7zip instead of regular zip or I could split
> the tests into smaller chunks too.

This looks very good, thanks!  Indeed, the archive size of almost one
Gigabyte is far too large for practical purposes.  We have now to find
solutions in the next month how to refine that.

What do you think about using a control file that specifies which
fonts to use, which glyphs in those fonts to use at which sizes,
modes, etc.?

I think some CSS magic to make the results more pleasing would also be
nice, but this is an extra of no particular importance currently.

> The report doesn't demonstrate the image comparison because there
> are obviously no regressions between my commit and master.  However,
> if there is one it will generate a page where you can see the
> differences between images on mouse over.  For text files it
> generates a diff report using pretty-diff. You can see this in the
> freetype-bench comparisons.

I *have* to see the image comparison live :-) When testing this
locally, I'll try to temporarily introduce a bug – since this depends
on the selected fonts I also have to do some trial and error to find a
good spot to do that.

> As I've said this should be mostly working now. I also believe I've
> commented the source pretty thoroughly.  I've also moved the scripts
> to a subdir as requested.  If there are any changes / improvements
> you would like to see to the reports or scripts please let me know.
> If not, I have listed several things I would like to do to polish it
> in my previous message.

The most important thing IMHO is that you (manually) prepare a subset
of the large report now (not larger than, say, 1MByte) so that other
interested persons can quickly download it to have a look.
Additionally, it might be helpful if you could send some screen shots
to the list.


    Werner