Re: [Gcl-devel] exec-shield mmap & brk randomization

[Camm Maguire]

Greetings, and thanks again for your helpful replies!

Roland McGrath <address@hidden> writes:
>
> The second layout change is what I suspect broke GCL.  It broke Emacs's
> unexec as well.  Personally, I consider this change incorrect.  However, we

Bingo.  In fact, we use exactly emacs' unexec.  I'm hoping your past
tense of the verb to break here indicates that a fix for unexec is
already at hand?

I've verified that this is where the segfault is, and if I understand
the following, likely the only such place.

> have not yet hashed out among the RH developers concerned with this area
> what the resolution will be.  Moreover, I tend to think that anything
> broken by it probably ought to be doing things differently in the long run.
> Since the dawn of time, the "break area" in Unix has started immediately
> after the executable's writable segment (i.e. after its .bss section) and
> extended upward from there.  By "the break area", I mean the region of
> memory starting at the address returned by sbrk the first time it's called
> after an exec.  From the beginning of Unix until two weeks ago Wednesday,
> the first `sbrk (0)' returned &end, the end of your .bss; increasing the
> break with sbrk calls gave a contiguous region from your data segment
> through to the current address of the dynamically-extended break.  In
> Fedora Core kernels, for processes in exec-shield mode, this is no longer
> the case.  The starting address of the break area is randomized at exec
> time, in a fashion similar to the randomization of mmap addresses.  The
> first call to `sbrk (0)' will tell you the lower bound of the region, which
> will not be lined up with the end of your executable's writable segment.
> As always, calls to sbrk to increase the size of the region will work as
> long as there is unused address space above the current break region
> (randomly placed mmaps will tend to be elsewhere, lower in the address
> space, and not impinge on break expansion).  But the exact location of the
> region is now somewhat unpredictable, and you can always expect a hole
> between static data (your program's writable segment, i.e. .data+.bss) and
> the dynamically-extended break region.
> 

Thankfully, GCL does not have to know precisely where its heap lower
bound is (apart from unexec) until the first call to sbrk(0) is made,
but we do absolutely need contiguity of this heap.  As long as you
don't dump brk altogether, I may retain my head of hair. :-).  We do
approximate this lower bound at compile time with a C define called
DBEGIN, but this is truncated to such a large unit that the
randomization does not cause problems, mostly.  There is some
probability that the DBEGIN determined at configure time even when
truncated is *greater* than the address returned by the first call to
sbrk(0), a definite no-no.  I'm sure we can work up some heuristic to
come up with a true lower bound for the range of randomized sbrk
addresses.

> As I said, I personally don't like this change.  That's because I consider
> starting at &end to have been part of the specification of the break
> functionality inherited from ancient Unix, and breaking such things is just
> plain wrong.  Nonetheless it is at least for the moment the way things are
> in FC1 kernels.  I don't want to engage in a discussion here about the
> merits of this change.  I would like to help hash out how precisely it
> affects you and any possible ways to work around it there might be.  (If
> figuring it all out turns out to lead you not to want to change anything
> and instead to complain heartily about the kernel behavior change, then
> that's as may be.)
> 

:-(.

> I said that anything broken by it probably ought to be doing things
> differently.  The reason I say that is that I consider the brk interface
> obsolete.  I can't really see any good reason to be using it in preference
> to the other options.  It's inherently limited as an allocation interface
> in that it provides just one contiguous region; address space fragmentation
> from the mappings for the executable, shared libraries, thread stacks, etc,
> will mean that many smaller discontiguous holes are available, and using
> only the break region will mean the total limit on useful allocation in the
> process can be much lower than the true limit imposed by the configured
> resource limits, the system implementation, or available virtual memory
> resources.  The other option to use mmap and be prepared to get
> discontiguous regions when requesting pages on separate occasions.  If
> contiguity is required, you can use the mremap call when available (AFAIK
> only on Linux kernels) to extend a region and move it if a sufficient
> contiguous region is not free in the original location.  If contiguity is
> only somewhat preferred but not strongly so, you can use a nonzero first
> argument to mmap (without using the MAP_FIXED flag), and you will
> ordinarily get the requested address or the next following address with a
> free region of the requested size.  (But note that there is still no
> guarantee of getting the requested region without MAP_FIXED and the robust
> program must be prepared to handle discontiguous regions.  FC1 kernels will
> in fact hand back predictable addresses, but future kernels might not
> always do so.)
> 
> If you specifically rely on treating static data and the dynamic break
> region as a single contiguous region, something has to change.  If you

Only in unexec.  If emacs has a fix, we can use it directly.

> don't rely on that, but just on knowing the start and end of the contiguous
> break region itself, then you can make the simple and portable change of
> using sbrk (0) instead of &end to initialize your idea of the lower bound
> of the break region.  In the long run, I would still recommend more drastic

This we do already, apart from unexec, thankfully.

> changes to avoid relying on a contiguous break region at all, for the
> reasons I gave in the previous paragraph.  If you make those changes, you
> will not impose so low a limit on the total memory you can allocate in a
> process, a benefit you'll get in the same way on all modern Unix-like
> systems (at least the 32-bit ones, I guess 64-bit systems always have more
> unused address space after the break than you could possibly need).
> 

I'm not really sure how much memory could be wasted, but this likely
seems a very small consideration compared to the complexity of
redesigning the garbage collector, etc.  GCL has static arrays of
length MAXPAGES used to mark various properties of the pages as they
are allocated, which in turn is indirected via a function of the
address and DBEGIN.


> Please let me know if there is anything I clarify or anything I can do to
> help you figure out what changes to your program might be best.
> 

Your comments are quite helpful.  It seems like if we can

1) fix unexec and 
2) come up with a configure time absolute lower bound to the first
sbrk after exec

we're gold.  Else we must

3) use setarch

Take care, 


> 
> Thanks,
> Roland
> 
> 
> _______________________________________________
> Gcl-devel mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/gcl-devel
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah