Re: GSOC Build tests

[Werner LEMBERG]

Hello Greg,


> figured out the cause. I have however been working on this as much
> as time allows. I've mostly hashed out the scripts to run regression
> tests using demos here:
> https://patch-diff.githubusercontent.com/raw/fundies/freetype2/pull/1.patch
> [...]

Some comments.

* There are too much top-level files.  Please move them to
  subdirectories.

* Please submit patches to the list that are generic enough to be
  included into 'master'.  This affects the CMake stuff, for example.

* If possible, don't use lines longer than 80 characters.

* Please avoid trailing whitespace.

> . The scripts still need a bit of work.  I've only just started the
>   bits that generate a html report for devs to inspect and I need to
>   hook it into azure pipelines.

My problem is that I have zero knowledge of azure.  So please...

> A couple other issues I want/plan to address are:
> 
> 1. I need to comment the scripts better

... do that in the first place.

> 2. I need to add a script argument to specify the demos commit as
>    not all commits of freetype2 are compatible with freetype2-demos
>    (in fact this broke on me over the past couple weeks)

Normally, 'freetype2' and 'freetype2-demos' stay in sync date-wise.
What about using the date of a given 'freetype2' commit to get a
corresponding commit in 'freetype2-demos'?

> 3. I currently run the demos in xvfb ...

Isn't `xvfb` going to be replaced with `xf86-video-dummy`?

>    ... but this isn't exactly compatible with windows so if you wish
>    for the regression test to run there I might need to modify some
>    the demos to just dump an image without opening a window

This might be generally useful, yes.  However, I suggest that you
concentrate on having the tests first.  It is rather unlikely that
FreeType itself behaves differently on Windows since it is quite
generic C code with only a very small amount of platform-specific
code.  In other words, getting screenshots of running FreeType demo
tools on Windows makes sense to catch problems in the GUI code of the
tools but probably not of the FreeType library.  For the time being I
think it is sufficient to have Windows compilation tests.

> 4. I don't see a clear way to get the number of characters in a font
>    using any of the existing demos so I might need to add a utility
>    for that but for now I just hardcode those values in each font
>    test

Please be careful with terminology: In general, you count the numbers
of *glyphs* in a font.  The number of character codes is strongly
dependent on a font's cmaps.  So I guess you want to know the number
of glyphs, right?

`ftdump` shows a 'glyph count' line.  Maybe this works for you.
Alternatively, a small tool to explicitly get this value should be
easy to write.

> 5. I'm not sure ftgrid and etc support all render modes?

Mhmm, maybe.  With `ftgrid`, for example, use keys 'F5' and 'F6' to
cycle through rendering modes.

Alexei, how do I specify keys like 'F5' with the `-k` command line
option?

> I'm currently working on the report generation as well as 1/2 and
> hope to have something to show soon.  However, to show results it
> would be helpful if someone could point me to a line of code I could
> tweak to subtly break rendering between commits so we can see it
> find faults.

It's not clear to me what you want to achieve.  Please elaborate.

> Also, if there are already any solutions to my issues with demos
> I've missed please let me know so I don't reinvent the wheel.

If my explanations above are not sufficient please don't hesitate to
ask more!


    Werner