> This looks very good, thanks! Indeed, the archive size of almost one
> Gigabyte is far too large for practical purposes. We have now to find
> solutions in the next month how to refine that.
Right now it runs ~12 fonts that have ~3k glyphs each; this is why it's so large. I think the best solution is splitting up runs of the tests to limit it to specific fonts and glyph ranges. That way you can download the smaller subset of tests that fail.
> What do you think about using a control file that specifies which
> fonts to use, which glyphs in those fonts to use at which sizes,
> modes, etc.?
The scripts are already basically structured to do this so adding that should be easy. If you could give me an idea of how you would like the config file structured I can write a parser easily enough.
> I think some CSS magic to make the results more pleasing would also be
> nice, but this is an extra of no particular importance currently.
I've never been the best at design. If someone can give me a design mockup or something similar they'd like it to look to though I can make it match fairly easily.
> The most important thing IMHO is that you (manually) prepare a subset
> of the large report now (not larger than, say, 1MByte) so that other
> interested persons can quickly download it to have a look.
> Additionally, it might be helpful if you could send some screen shots
> to the list.
Here is a small subset of the glyphs. I've manually modified one in gimp to show the image diffing.
Here are some images:
On Thu, Jul 30, 2020 at 4:39 AM Werner LEMBERG <
wl@gnu.org> wrote:
> I've finished the core of the regression tester.
Great!
> You can now run it and generate a html report but you will need a
> few tools installed: imagick, xwd, npm, pretty-diff and xvfb. This
> should all be listed in ft-regression.sh's comments.If you want to
> run it locally make sure you have a ~/test-fonts with .ttf files in
> it.
Thanks; I will try that in the next few days, hopefully.
> This is basically what I wanted to accomplish for my GSOC project
> but I can do a lot of the things mentioned in my previous message if
> I pass my evaluation.
:-)
> You can download the report it generates from here:
> https://dev.azure.com/fundies/9eabb07a-6a4d-4b68-b22e-60f9e02c1927/_apis/build/builds/232/artifacts?artifactName=Archlinux%20Regression%20tests&api-version=5.1&%24format=zip
> The file is rather large. In the future, I could probably shrink it
> quite a bit by using 7zip instead of regular zip or I could split
> the tests into smaller chunks too.
This looks very good, thanks! Indeed, the archive size of almost one
Gigabyte is far too large for practical purposes. We have now to find
solutions in the next month how to refine that.
What do you think about using a control file that specifies which
fonts to use, which glyphs in those fonts to use at which sizes,
modes, etc.?
I think some CSS magic to make the results more pleasing would also be
nice, but this is an extra of no particular importance currently.
> The report doesn't demonstrate the image comparison because there
> are obviously no regressions between my commit and master. However,
> if there is one it will generate a page where you can see the
> differences between images on mouse over. For text files it
> generates a diff report using pretty-diff. You can see this in the
> freetype-bench comparisons.
I *have* to see the image comparison live :-) When testing this
locally, I'll try to temporarily introduce a bug – since this depends
on the selected fonts I also have to do some trial and error to find a
good spot to do that.
> As I've said this should be mostly working now. I also believe I've
> commented the source pretty thoroughly. I've also moved the scripts
> to a subdir as requested. If there are any changes / improvements
> you would like to see to the reports or scripts please let me know.
> If not, I have listed several things I would like to do to polish it
> in my previous message.
The most important thing IMHO is that you (manually) prepare a subset
of the large report now (not larger than, say, 1MByte) so that other
interested persons can quickly download it to have a look.
Additionally, it might be helpful if you could send some screen shots
to the list.
Werner
