suspected use-after-free leads to bootstrap failure

[Jim Meyering]

Jim Meyering wrote:
> Eli Zaretskii wrote:
>>> From: Jim Meyering <address@hidden>
>>> Date: Wed, 30 Mar 2011 09:42:42 +0200
>>> Cc: address@hidden, address@hidden
>>>
>>> This command,
>>>   env MALLOC_PERTURB_=0 MALLOC_CHECK_=0 make -j9 bootstrap
>>> has succeeded for me on each of the last three mornings (Mar 28-30).
>>>
>>> I manually set those two MALLOC_*_ variables to 0 because
>>> when I don't, emacs fails to bootstrap.
>>
>> It's a pity this problem was not reported to the bug tracker.  (At
>> least I couldn't find it; apologies if I missed it.)
>
> I thought I reported it to some emacs development list months ago,
> but a quick search didn't find it.
>
>>> I suspect that emacs is using free'd memory containing
>>> values that would normally be unoffensive, but when you set
>>> those envvars (esp MALLOC_PERTURB_) to nonzero, it makes
>>> glibc scribble on free'd buffers, and that makes emacs
>>> exhibit an actual failure.
>>
>> Can you use bisect to find the guilty commit?
>
> Finding a commit for which a perturbed "make bootstrap" succeeds
> was a challenge.
>
> I bootstrapped 8 or 10 times, going back to 2009
> in steps of 500, then 1500 commits.  Same failure
> each time, until I started getting link errors:
>
>   /usr/bin/ld: xftfont.o: undefined reference to symbol 
> 'XRenderQueryExtension'
>   /usr/bin/ld: note: 'XRenderQueryExtension' is defined in DSO 
> /usr/lib64/libXrender.so.1 so try adding it to the linker command line
>   /usr/lib64/libXrender.so.1: could not read symbols: Invalid operation
>
> I worked around that by inserting -lXrender into the generated Makefile:
>
>     perl -pi -e 's/(-lfreetype )/$1-lXrender /' src/Makefile
>
> With that, I finally found a successful build at this git commit:
>
> commit 84655cfe88efb24c256302d016cd037d22544cca
> Author: Stefan Monnier <address@hidden>
> Date:   Fri Nov 6 18:47:48 2009 +0000
>
>     Let integers use up 2 tags to give them one extra bit and double their 
> range.
>     * lisp.h (USE_2_TAGS_FOR_INTS): New macro.
>     (LISP_INT_TAG, case_Lisp_Int, LISP_STRING_TAG, LISP_INT_TAG_P): New 
> macros.
>     ...
>
> Maybe someone else will do the actual bisection:
>
>     Bisecting: 4164 revisions left to test after this (roughly 12 steps)
>
> This is the command to run:
>
>     env MALLOC_PERTURB_=44 MALLOC_CHECK_=3 make -j9 bootstrap
>
> If not, I'll get to it, eventually.

I did that.
This is the offending commit:

5a98a2a69b1a15173ce4bfa53307608a7150b407 is the first bad commit
commit 5a98a2a69b1a15173ce4bfa53307608a7150b407
Author: Stefan Monnier <address@hidden>
Date:   Sun Apr 18 17:49:33 2010 -0400

    Hash-cons pure data.
    * alloc.c (Fpurecopy): Hash-cons if requested.
    (syms_of_alloc): Update purify-flag docstring.
    * loadup.el: Setup hash-cons for pure data.

Bootstrapping (with MALLOC_PERTURB_=44 MALLOC_CHECK_=3) from there,
I get this failure:

    In toplevel form:
        /e/emacs/lisp/language/thai-word.el:10738:5:Error: \
          Memory exhausted--use C-x s then exit and restart Emacs

Bootstrapping from the one just before succeeds.