[Gcl-devel] Re: more on *after-gbc-hook*

[Camm Maguire]

Greetings!

"Warren A. Hunt Jr." <address@hidden> writes:

> Hi Camm,
> 
> >  Greetings!  While I cannot reproduce this exactly (your image appears
> >  tighter on memory than mine), it would appear that what is happening
> >  is that the hole is being overrun, sgc turned off, a gc collecting
> >  relocatable pages triggered to move them and make a new hole, then sgc
> >  turned on again, which triggers another gc in getting enough space for
> >  the sgc pages.  So before the original call to alloc_page completes,
> >  two gc calls have been made, and the pending after-gbc-hook flag set
> >  to trigger (only one) call to this function when it is safe.
> 
> >  In general, it appears that the existing design strove to first
> >  satisfy the allocation request which triggered the gc, and then call
> >  the hook.  If the hook is called when the allocation is partially
> >  complete, then the hook itself could trigger another allocation which
> >  would corrupt the partial one.  The hook could in principle be called
> >  at the end of GBC, but if it allocated enough space to trigger another
> >  gc one could get endless recursion.  This also could happen if the
> >  original allocation is not first satisfied -- at least if it is, the
> >  image is guaranteed to be making some other progress before calling
> >  the hook. 
> 
> >  In any case, with this design, these two gc calls are not separated by
> >  a safe place to call the hook, if I understand correctly.
> 
> >  I guess I don't understand the purpose of the hook.  Do you just want
> >  one call per gc reasonably close to the end of the call even if two
> >  pile up together, or do you want a call on a pristine swept heap
> >  before the next allocation is made?
> 
> Thanks for your message, as it clears up in my mind something we have
> seen previously.  Quite a few times, when pushing the allocation of
> memory pages up near the maximum available, Bob and I have seen the
> garbage collection process simply print GC messages one after another
> without end.  Perhaps we have been in this "endless recursion".
> 

If you discover a reproducible way to trigger same by happen-stance,
please send it my way.

<unique cons implementation description snipped>

OK, I am going to try to move this call to the end of GBC and rely on
the user to write the hook in such a way as to avoid endless
recursion.  It looks possible.

In general, however, it appears that what you want are weak
references.  I.e. lists, arrays, or hash-tables which do not cause
their contents to be marked.  Then you can use, e.g. normal object
arrays, which could even be non-static for speed (i.e. the pointer to
the cons body is stored in the array, the cons body itself on normal
cons pages as usual), and rely on the existing sweeper to sweep any
unused elements.  One then simply has to follow the sweeper by an
immediate traversal of the weak object (e.g. array), and set any freed
pointers therin to something special meaning 'available',
e.g. OBJNULL.  This can either be done in a hook as you have it, or in
some special code we could add to gbc.c.  I.e. si::*weak-objects*
might be a list of said special container objects, and a flag bit
could be added to struct array et. al. and hash_table to designate
that the marker should skip the contents when traversing this object.

OK, just found a moment and moved call_after_gbc_hook to the end of
GBC.  Looks ok for the simple tests we've been doing.  Please test,
and let me know if you can break it.  If not, I'll remove the
commented out older invocations still in the code at present.

Take care,

> 
> Again, thank you so much for all of your work on GCL.  Your work
> enables our current approach and many other things Bob and I are
> pursuing.  We certainly appreciate all that you do for GCL.
> 
> Cheers,
> 
> Warren
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah