[MIT-Scheme-devel] Executable memory: some apps that work on RH9 don't on FC1

[Chris Hanson]

   Date: Mon, 17 Nov 2003 12:50:11 -0800
   From: Roland McGrath <address@hidden>

   When MAP_FIXED is passed (fixedp!=0), the mapping will overwrite
   any other mappings that exist.  So if the address range overlaps
   some shared libraries or something like that, it will clobber that
   part of the address space and who knows what could happen.  In
   older kernels, shared libraries would always end up in a high part
   of the address space, so assuming a huge low region was available
   worked.  Now shared libraries (and any mmap region) are more likely
   to be located at random addresses that may be in the low part of
   the address space.  It has never been safe or kosher to assume some
   large part of the address space would never be used for shared
   libraries.  cscheme needs to change its plan for calling mmap.  If
   you need a big contiguous region of address space into which you
   will place multiple separate mappings, then the only safe thing to
   do is to mmap a region of the whole needed size without MAP_FIXED
   (e.g. using PROT_NONE), and then overwrite portions of that mapping
   with MAP_FIXED mappings to get the layout you want.

I appear to be missing some of the context of this discussion.  What
kernels have this behavior?  I have run MIT/GNU Scheme on Debian
unstable using many different 2.4 and a handful of 2.6-test kernels,
and I haven't seen this problem.

That mapping code dates back to the days of libc5.  At that time, if
you tried to map some space that overlapped a shared library, you got
an error.  Obviously that's changed.  Recently there hasn't been an
issue because all static and shared-library memory allocations have
been mapped into the top half of the address space.

But the problem remains that this code doesn't do what it's supposed
to.  Is there some way for me to probe the address space to find the
largest hole in a given region of the space?  (Not using signal
handlers to catch SIGSEGV traps!)

Here's some technical background: MIT/GNU Scheme uses a high-tag
object representation, with 6 bits of tag.  Addresses are converted to
objects by ORing the type code onto the address, and the reverse
operation simply ANDs the type code away.  Due to this representation,
and to other characteristics of our implementation, we need the
largest possible block of space under 2^26 -- ideally all of it.

It's possible to modify the implementation to handle a block outside
of this space, but the block still needs to be contiguous.  The cost
of the modification is a one- or two-instruction overhead for each
address/object conversion.  Since these conversions happen in _many_
places, this will have a significant effect on program size, which
will increase pressure on the already too-small address space.  There
will also be a small but pervasive time penalty.

Faced with this situation, I'll probably just statically link the
program.

IMO it's a bug that the application doesn't have any control over its
address space.  I should be able to tell the linker and loader that
certain areas are reserved.  That's possible with the linker, but I'm
not aware of any way to control the loader.

So why does the kernel care where shared libraries are mapped?  Why is
the application working for the kernel instead of the other way
around?