obstack usage question

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've noticed that M4 currently has a rather wasteful ratio of heap
allocation to usage based on its use of obstacks.  Obstacks make a great
structure for collecting an arbitrary amount of text whose size is not
known in advance, as is the case with m4 macro invocations/expansions.
But in m4, since macro invocations can be nested, and the inner macro
invocation add text to the outer macro's argument, the code currently
calls obstack_init() for every macro invocation so that multiple
invocations can independently grow their expansions.  Thus, in the typical
case, a 4096-byte chunk is allocated for the obstack of every macro
invocation, even though many macros expand to something relatively short
in length (ie. less than 100 bytes), for a huge overhead of unused heap
space when processing nested macros.

It would be nice if I could reuse the same obstack for multiple nested
calls; it would result in much less space overhead (packing the processing
of the nested calls into the unused space of the outer call, rather than
calling malloc again for a new chunk on every call), and less time
overhead (fewer calls to malloc).  My only problem in doing this is that
there doesn't seem to be an efficient way to suspend and resume the
creation of a growing object on an obstack.  Right now, I must do
something like obstack_finish on the current growing object, process the
nested call, copy the finished partial object out to temporary storage,
call obstack_free back to the base of the incomplete object, then call
obstack_grow to get the temporary storage copied back to a growable
object.  But that double copy is bothering me; I would really like to
reopen a finished object for further growth without having to use
temporary storage.  Any thoughts on the matter?  Is this worth taking up
with glibc?

- --
Life is short - so eat dessert first!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFKpxC84KuGfSFAYARAjxEAJ9AZ4CnlGFNGW+OZKAtDTbVZPzbGQCeJfcQ
JCX2sekv1pdgFR2BlcKuo7U=
=TD7O
-----END PGP SIGNATURE-----