Re: possible reason for slowness with Guile 2.2?

[Han-Wen Nienhuys]

On Sat, Jul 25, 2020 at 9:03 PM Jonas Hahnfeld <hahnjo@hahnjo.de> wrote:

> Am Samstag, den 25.07.2020, 21:01 +0200 schrieb Jonas Hahnfeld:
> > Continuing the discussion from
> > https://gitlab.com/lilypond/lilypond/-/merge_requests/225: One of the
> > remaining reasons for slower execution times with Guile 2.2 appears to
> > be the output framework. Comparing the runtime of Paper_book::output
> > for MSDM.ly shows around 1 second for Guile 1.8, but more than 7
> > seconds with Guile 2.2, from a total of ~30 seconds (Guile 2.2). Note
> > that this includes the time for Ghostscript!
> >
> > After looking into some profiles, I think this is in large parts due to
> > Paper_outputter::output_stencil which eventually calls scm_eval
> > from Paper_outputter::scheme_to_string. That's nothing new and has been
> > suspected to be the reason. However, Han-Wen reports that switching to
> > Guile 3.0, which is supposed to significantly improve the performance
> > of scm_eval, does not change much and it's still slower than Guile 1.8.
> >
> > As far as I currently understand, LilyPond calls scm_eval for every
> > stencil which might be many (several thousands per page for MSDM.ly, up
> > to ~4000 for a very crowded score of mine). To verify, I wrote attached
> > micro-benchmark that calls scm_eval for the expression (+ 1 2)
> > 1,000,000 times. Compiled with $ gcc -O2 -Wall eval.c -o test -lguile,
> > I see ~2.7s for Guile 1.8. With -lguile-2.2 this explodes to 36s! I
> > don't have an installation of Guile 3.0 around (please share if you
> > have), but would expect similar results.
>
> Nah, forgot the attachment... here we go.
>

For a representative benchmark, you have to create a list with a symbol and
int literals; your benchmark also measures parsing time.

The output framework is one of the oldest pieces of Scheme infrastructure
(if you follow its roots, you can find it in embryonic form in
commit bac78d4e469b6f5612b51536165b0855284321ab from 1998). Using eval to
translate a (symbol args) expression seemed like a good alternative to the
symbol dispatch that the aforementioned commit had, but it isn't how I'd
build it today.

(Fun fact: lilypond once sported an ASCII art backend; look for AsciiScript
in the git history.  Adding Jan just to have another old fogey reminiscing
of our lore)

The question though is: what is a good interface? Existing code that
interprets stencils look like a object with state (eg. the drawing
context), and then do a dispatch on what symbol they encounter (see eg.
interpret_stencil_for_skyline)

The recent discussion around making --safe the default are also relevant:
if we split lilypond in two parts (the untrusted renderer and the trusted
part that handles PDF/PNG conversion), we'd likely have to reorganize how
Stencil expressions end up in PS files.

-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen