Re: [Chicken-hackers] Heap size calculation in rereclaim2

[Peter Bex]

On Wed, Oct 14, 2015 at 06:48:19PM +0200, "Jörg F. Wittenberger" wrote:
> I'm not so sure.  I had problems with chicken running out of memory upon
> startup years ago.  (Which I eventually worked around by triggering
> additional gc's.  I might try to figure the details out later if needed.)

Additional GCs may empty the stack which may prevent the heap overflow,
because we also request slightly too much for symbol allocation (10 words
instead of the requisite 7 words).

> >  And my guess is that most large literals are strings or blobs (or
> > srfi-4 vectors which are internally blobs).  These do not trigger this
> > problem due to the fact that they're allocated with malloc().
> 
> Wait - when are those blobs/strings allocated with malloc()?  I know
> it's possible if explicitly requested.  But I rarely ever did so.

We're _always_ using malloc for string & blob *literals* in compiled code.
So if you compile for example (define foo "bar"), then both the strings
"foo" and "bar" are going to end up in malloc()ed memory.

For foo, the symbol itself is allocated in the CHICKEN heap, but its
string pointer goes to the malloc() heap.

Just look at decode_literal2.  Search for C_STRING_TYPE and see
the code of C_static_string(), which is also called from C_h_intern_in().

> Otherwise I did abuse strings to binary data of several 10 MB, assuming
> this would overwhelm the GC.  Was surprised it did not.
> 
> So if there's magic to automagically switch to malloc() for huge
> blobs/string than forget this message.  Otherwise your guess may be wrong.

There's no magic.  We *always* allocate static literals with malloc(), no
matter how small.  When you call (make-string) or otherwise dynamically
build the string it will be allocated in the normal way.

Now, typing all this, I'm starting to wonder why we don't just use the
pointer to the static memory allocated from .DATA to avoid duplicating
it at all?  It is allowed by the spec to make them use shared storage.
It would save a small amount of overhead in copying, but it can
potentially save a lot of wasted memory, depending on the program.

Cheers,
Peter

signature.asc
Description: Digital signature