bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[David Kastrup]

Andy Wingo <address@hidden> writes:

> On Thu 23 Jun 2016 18:59, David Kastrup <address@hidden> writes:
>
>> I don't see anything protecting sym_big or sym_little (more accurately,
>> 'big or 'little which are non-immediate SCM values) from collection
>> which would make sym_big and sym_little useless for comparison.
>>
>> I'm assuming that not the whole bss segment is getting scanned by
>> BoehmGC.
>
> The whole bss segment is scanned by the GC :)  See the third paragraph
> here.
>
>   
> https://www.gnu.org/software/guile/docs/master/guile.html/Garbage-Collection.html#Garbage-Collection
>
> This section applies to 2.0 as well.

Sigh.

    In practice, the wasted memory seems to be no problem, as the static
    C root set is almost always finite and small, given that the Scheme
    stack is separate from the C stack."

"almost always" means "for trivial applications and/or applications only
written in Scheme".  But Guile is marketed as an _extension_ language.

LilyPond has a table-driven lexer and parser and quite a bit of other
static data.

A quick peek with ld shows

.bss            0x0000000000aa2340    0x21218
.data           0x0000000000aa1080     0x12a0

Ok, .data seems absurdly compact for a table-driven parser with
something like 800 states alone, let alone all the rest.  So I might
have problems interpreting the output.

At any rate, .bss does not look exactly small.

Obviously, I am not overly happy with the design choices and rationale
taken for Guile-2.x memory management.

    These “false positives” might keep SCM objects alive that would
    otherwise be considered dead. While this might waste memory, keeping
    an object around longer than it strictly needs to is harmless.

Except that a single object may keep a whole lot of other stuff alive.
LilyPond is usually called once for compiling literally thousands of
files in one session sequentially.  It makes sure to call garbage
collection explicitly between files, when the return stack is indeed
guaranteed to be quite small.

Using Guile-1.8 we actually track non-freed objects at that time in our
testing versions (by setting a flag that will let the mark hooks
complain for object types that should not be around at that time).  It
does deliver false positives but at a pretty moderate rate: I'd guess
about 5% or so of complete runs through the regression test database are
affected.  We won't be able to do these tests with Guile-2 anyway since
they format output in the marking phase and that crashes the GC
subsystem (it's likely not kosher in Guile-1 either but the GC character
is more sequential there).

I doubt that the false positive rate would stay that low when
conservatively marking all the data segments rather than just the stack
at a known low mark.

And I don't really see how to optimize for application layout patterns
_not_ matching Guile's assumptions which again takes us to the question
how much an extension language should determine the design of the main
application.

Obviously, I have no problem with Guile deciding to conservatively scan
the libguile data segments: the layout and size of those are under its
own control.

Either way, sym_big and sym_little seem to be covered.

-- 
David Kastrup