[Chicken-hackers] [PATCH][5] Make temporary stack resizable (fixes #1098)

[Peter Bex]

Hi all,

Now that we're using argvectors, we're no longer limited by any arbitrary
C macros or platform-specific hacks.  The only two things that determine
the number of arguments that can be passed to a procedure are:

1) The size of the C stack (because that's where argvectors are allocated)
2) The size of the temporary stack (because that's where argvectors are
   copied for safekeeping during GC, which blows away the stack)

For now, fixing 1) is a bit of overkill, though it might be possible by
allocating argvectors in the heap.  Doing this will be tricky because now
we're assuming everywhere that the stack is to be used for argvectors.

But 2) has been a thorn in my side for a long time.  It gets dangerously
close to the "arbitrary limitations" that the Scheme way of life always
tries so hard to avoid.  Especially since the temporary stack can easily
be resized!

So, here we go, a patch to dynamically resize the temporary stack.
It does so only during save_and_reclaim, not in C_save() or C_rescue(),
because I think those aren't used for very large argument counts, so they
won't stress the temporary stack (correct me if I'm wrong).  I've only
implemented it for the chicken-5 branch due to the relience on the
C_ilen() function.  If it is important enough, we could port it to master
by using log2() or simply avoid attempting to scale to powers of 2.

I wasn't sure what to do with ##sys#apply-argument-limit, so for now
it simply returns half the stack size because the entire stack size
seemed excessive and incorrect.  However, the stack _does_ control
the effective maximum number of arguments.

The test is also simplified a little.  Initially I tried hard to make
a foreign inline function which forced a GC via save_and_reclaim, but
in the end I decided it's much simpler to set a smaller initial size
for the temporary stack: this means the resize will be triggered (this
can be seen now because I've added -:d to that program in runtest.sh).

Lowering the initial size also means that every CHICKEN program will
use 12k or 24k less memory (depending on word size), which is nice.
Only in case of excessive argument passing will the memory use increase,
but only temporarily!

We could set the default to an even lower value, like 256, 64 or even 16.
In my experience, the average argument count that get saved in a normal
program will be below 10 anyway.  We would probably like to avoid
unnecessary malloc()s, so such an extremely low value is not desirable,
but we could add some more magic to the resizing algorithm if necessary.

This patch tries hard to avoid too many resizing events.  Perhaps this
is silly and should be simplified: if the stack isn't the default size,
and the required size is less, we could simply resize it to that size
again.

PS: I think cconv-sample.c can be removed now too, as it was only useful
as a basis for new apply-hacks.  What do y'all think?

Cheers,
Peter

0001-Dynamically-resize-temporary-stack-when-needed.patch
Description: Text Data

signature.asc
Description: Digital signature