[Gcl-devel] malloc() and the inner workings of alloc.c

[Edward Catmur]

Hi,

I'm trying to debug a segfault that happens in saved_ansi_gcl when using
Gentoo's libsandbox as an LD_PRELOAD wrapper. libsandbox is used during
compilation and install to wrap filesystem calls and error out when the
process attempts to touch anything outside the compilation, staging
install, home or tempfile directories.  It's not a security measure,
rather for ensuring reproducible, packageable builds.

The segfault is at http://bugs.gentoo.org/show_bug.cgi?id=161041 (please
ignore the discussion of RESTRICT and FEATURES); also reproduced at end
below[1].

libsandbox of course needs to use malloc() and does so through the PLT,
so when wrapped around saved_ansi_gcl it's using that provided by
alloc.c.  What appears to be happening is that the malloc arena pointers
are being incremented past the end of the data segment without
corresponding sbrk() calls.  I logged libsandbox's malloc calls and
found they tend to be quite large; most are for 2048 bytes.

I tried a voodoo-style patch (alloc-sandbox-fix.patch on the Gentoo bug;
attached) and was very surprised when it seemed to work.  If there's
someone on list who understands alloc.c, would you mind checking to see
whether the patch is correct, or just covering up an underlying problem?
Alternatively any help in understanding alloc.c would be appreciated,
particularly in what core_end and heap_end are for, and how they should
be adjusted when a malloc() call requires increasing the data segment
size.

Many thanks,

Ed


1. Backtrace from segmentation fault
Program received signal SIGSEGV, Segmentation fault.
0x4034d486 in __realpath (name=0x8fff000 "/var/db/aliases.db", 
    resolved=0x9001000 <Address 0x9001000 out of bounds>) at canonicalize.c:98
98            rpath[0] = '/';
#0  0x4034d486 in __realpath (name=0x8fff000 "/var/db/aliases.db", 
    resolved=0x9001000 <Address 0x9001000 out of bounds>) at canonicalize.c:98
#1  0x4001f646 in init_env_entries (prefixes_array=0x400273fc, 
    prefixes_num=0x40027400, env=0x400238db "SANDBOX_PREDICT", 
    prefixes_env=0xbfb69e22
"/var/tmp/portage/dev-lisp/gcl-2.6.7-r2/homedir/.:/usr/lib/python2.0/:/usr/lib/python2.1/:/usr/lib/python2.2/:/usr/lib/python2.3/:/usr/lib/python2.4/:/usr/lib/python2.5/:/usr/lib/python3.0/:/var/db/ali"...,
 
    warn=1) at sandbox-1.2.18.1/src/libsandbox.c:1064
#2  0x4002088c in before_syscall (func=0x400237e1 "open_rd", 
    file=0x8394440
"/var/tmp/portage/dev-lisp/gcl-2.6.7-r2/image/usr/lib/gcl-2.6.7/unixport/temp")
at sandbox-1.2.18.1/src/libsandbox.c:1514
#3  0x4002197e in open_DEFAULT (
    pathname=0x8394440
"/var/tmp/portage/dev-lisp/gcl-2.6.7-r2/image/usr/lib/gcl-2.6.7/unixport/temp",
flags=<value optimized out>)
    at sandbox-1.2.18.1/src/libsandbox.c:1551
#4  0x08088ac2 in unexec (
    new_name=0x9001000 <Address 0x9001000 out of bounds>, 
    old_name=0x1003 <Address 0x1003 out of bounds>, data_start=1073902592, 
    bss_start=0, entry_address=0) at unexelf.c:672
#5  0x08089d3e in Lsave () at save.c:12
#6  0x0805293d in siLsave_system () at main.c:977
#7  0x080d8353 in eval (form=0x8522800) at eval.c:1090
#8  0x080d85e5 in fLeval (x0=0x8ec7dc8) at eval.c:1178
#9  0x08056c70 in IapplyVector (fun=0x853de24, nargs=1, base=0x839b5e4)
    at nfunlink.c:229
#10 0x080d8b09 in funcall (fun=<value optimized out>) at eval.c:190
#11 0x0817e3a7 in LI1 () at gcl_top.c:140
#12 0x080d7743 in quick_call_sfun (fun=0x853d000) at eval.c:117
#13 0x080d8a64 in funcall (fun=<value optimized out>) at eval.c:178
#14 0x08056cc7 in IapplyVector (fun=0x853d000, nargs=0, base=0xbfb6809c)
    at nfunlink.c:239
#15 0x080d75e5 in fLfuncall (fun=0x853d000) at eval.c:1140
#16 0x08056c70 in IapplyVector (fun=0x853de4c, nargs=1, base=0x839b5b4)
    at nfunlink.c:229
#17 0x080d8b09 in funcall (fun=<value optimized out>) at eval.c:190
#18 0x080d822d in eval (form=0x8522800) at eval.c:1092
#19 0x080d8827 in funcall (fun=<value optimized out>) at eval.c:327
#20 0x080d822d in eval (form=0x8522800) at eval.c:1092
#21 0x080d8827 in funcall (fun=<value optimized out>) at eval.c:327
#22 0x08053324 in main (argc=1, argv=0x0, envp=0x0) at main.c:373

alloc-sandbox-fix.patch
Description: Text Data