libio/libioP.h:ALLOC_BUF() unconditionally uses mmap()

[Wayne Whitney]

Dear glibc folks,

I have been having a problem on ia32 Linux with a program (MAGMA) linked
against glibc that eschews malloc() and does its own memory allocation
with sbrk().  Since brk() on ia32 Linux grows the data segment upwards
from circa 0x08000000, and by default mmap() starts allocating at
0x40000000, MAGMA will run out of memory after a bit under 896MB if there
are any persistent mmap() mappings.  To that end, MAGMA is statically
linked to avoid the use of mmap() by the dynamic loader, but there is
still one persistent mmap() mapping:

  address@hidden whitney]$ cat /proc/`pidof magma`/maps 
  08048000-08afb000 r-xp 00000000 21:07 64318 magma 
  08afb000-08c3e000 rw-p 00ab2000 21:07 64318 magma 
  08c3e000-0bc54000 rwxp 00000000 00:00 0 
  40000000-40001000 rw-p 00000000 00:00 0 
  bfffd000-c0000000 rwxp ffffe000 00:00 0

I brought this issue up on the Linux Kernel Mailing List, and it was 
suggested that this was due to a misbehavior by glibc.  Specifically:

On 13 Dec 01, Petr Vandrovec <address@hidden> wrote:

> [The mapping comes from] code in glibc-2.2.4/libio/libioP.h:ALLOC_BUF(), 
> which unconditionally does 'mmap(0, ROUND_TO_PAGE(size),
> PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)' instead of
> 'malloc(size)' when it finds that underlying system supports malloc.

So I wanted to ask if this use of mmap() could be changed to a use of
malloc() without any ill effects?  Normally I would just try something
like this out myself, but modifying glibc seems pretty scary.  :-)

If it is possible to change this mmap() to a malloc(), then MAGMA can set
the environment variables that tune glibc's malloc() to tell malloc()  
never to use mmap().  That should eliminate this mapping at 0x40000000.  
Then as long as it is OK for a program to use sbrk() in conjuction with
glibc's internal use of malloc(), MAGMA should be able to happily brk() up
to the top of user space (rather, the top of the stack).

Thank you for your time.

Best wishes, Wayne