Re: XEmacs 21.4.10 crashes with glibc 2.3.1

[Stephen J. Turnbull]

>>>>> "Wolfram" == Wolfram Gloger <address@hidden> writes:

    >> First, your patch also affects "portable dumper" builds which
    >> build and (mostly) run fine.  Is this intentional?  Ie, is this
    >> a generic problem with our allocator implementation, which
    >> "just happened" to manifest dramatically only in unexec builds
    >> on very recent glibcs?

    Wolfram> That's quite probable.  You see, the problem is Lisp's
    Wolfram> tagged pointers.  Pointers to Lisp objects are "coloured"
    Wolfram> in their most significant bits (I think 3 bits) with type
    Wolfram> information.

Not in XEmacs.  In XEmacs >= 21.4, there are 3 kinds of immediate Lisp
Objects and a pointer Lisp Object, and the tags are in the _low_ two
bits.  If the low bit is 1, it is a 31-bit Lisp integer.  If the two
low bits are 10, the object is a 30-bit Lisp character.

If the two low bits are 00, then the object is an lrecord type, and
the bit pattern is interpreted as (struct lrecord *).  Note that there
is no difference between the bit pattern as a Lisp Object and the bit
pattern used to address the record.

This is the default implementation of Lisp Object in 21.1 as well, but
I believe that 21.1 still has the option to use the traditional high
tag implementation.  Here's the #define from 21.1'a config.h.in:

/* If defined, use a minimal number of tagbits.  This allows usage of more
   advanced versions of malloc (like the Doug Lea new GNU malloc) and larger
   integers. */
/* --use-minimal-tagbits  */
#undef USE_MINIMAL_TAGBITS

    Wolfram> You had the M_MMAP_THRESHOLD set to 64k, a reasonable
    Wolfram> choice IMHO; one Lisp vector allocation from the "temacs
    Wolfram> -dump" run just exceeded that threshold.

Well, obviously we were doing something wrong, because your patch
fixed it.  But it seems we understood the problems with traditional
high tags and Doug Lea malloc.

    Wolfram> IMHO the autoconf test should check whether
    Wolfram> mallopt(M_MMAP_MAX) is available, and deduce that it's a
    Wolfram> variant of Doug Lea malloc.

FWIW, we actually use the following test, based on malloc_set_state
and __after_morecore_hook.  I have never heard that it made a mistake,
either false positive or false negative.

if test "$with_dlmalloc" != "no"; then
        doug_lea_malloc=yes
else
        doug_lea_malloc=no
fi
after_morecore_hook_exists=yes
AC_CHECK_FUNC(malloc_set_state, ,doug_lea_malloc=no)
AC_MSG_CHECKING(whether __after_morecore_hook exists)
AC_TRY_LINK([extern void (* __after_morecore_hook)();],[__after_morecore_hook = 
0],
  [AC_MSG_RESULT(yes)],
  [AC_MSG_RESULT(no)
   after_morecore_hook_exists=no])

dnl Tests to get the value of GNU_MALLOC go here.

if test "$doug_lea_malloc" = "yes" -a "$GNU_MALLOC" = "yes" ; then
  GNU_MALLOC_reason="
    - Using Doug Lea's new malloc from the GNU C Library."
  AC_DEFINE(DOUG_LEA_MALLOC)
  if test "$after_morecore_hook_exists" = "no" ; then
    GNU_MALLOC_reason="
    - Using Doug Lea's new malloc from the Linux C Library."
    AC_DEFINE(_NO_MALLOC_WARNING_)
  fi
fi



-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.