Re: linuxthreads/malloc problem

[Charles G Waldman]

Ulrich:  

I am not claiming this is a glibc bug.  This is a topic which came up
for discussion on this list and I am following up on Wolfram's request
for more information.  If this discussion is inapproprate for this
list we will be pleased to take it elsewhere.  But it is my impression
that this topic is of interest to many of the readers of this list.

Wolfram & everybody else on the list: 

I am CC'ing this to Tim Peters and Jeremy Hylton who are core Python
developers (I am just a part-time volunteer).  These guys were heavily
involved in the discussions of this problem back in July/August.

Tim, Jeremy:

There were some comments on the glibc-bug mailing list talking about
pthreads misbehavior possibly due to Linux kernel bugs (?) and of
course it reminded me of the thread/fork problems we were having with
Python - some people on this list are interested and have asked for
more information about the bug we `fixed'.



Wolfram Gloger writes:

 >  > If it is not the above `problem', I'd appreciate a pointer to the
 >  > Python sources (without the workarounds).

Here's a message from Jeremy discussing the problem:

http://www.python.org/pipermail/python-dev/2000-July/014376.html

And Tim's analysis:

http://www.python.org/pipermail/python-dev/2000-July/014410.html

There are many other messages in the python-dev archives in July and
August 2000 discussing this issue.    Look for the words "test_fork1"
and "thread".  

Note that we *expect* this program to fail - it forks and the child
processes do not correclty clear mutexes inherited from the parent -
but the thing that confused everybody was the exact *way* that it
failed - the hangs were not in the child processes but in the parent.
As Tim Peters wrote:

 > It's a baffler!  AFAIK, nobody yet has thought of a way that a fork
 > can screw up the state of the locks in the *parent* process (it
 > must be easy to see how they can get screwed up in a child, because
 > two of us already did <wink>).
 > But your hangs appear to be in parents, as the gdb stacktrace shows that a
 > parent process is (presumably) hanging while trying to reacquire the global
 > lock...

And here is the message where I propose the kludge^H^H^H^H^H^Hworkaround
which was eventually accepted for Python 2.0:

http://www.python.org/pipermail/python-dev/2000-August/015270.html

The easiest way to replicate this is probably to grab a copy of the
Python 1.6 sources from
ftp://ftp.python.org/pub/python/src/python-1.6.tar.gz ; do
"./configure --with-thread; make" and use the resulting Python
interpreter to run Jeremy's test program , which I attach here for
convenience:


------cut here-------
import os
import thread

def f():
    while 1:
        if os.fork() == 0:
            print "%s %s" % (thread.get_ident(), os.getpid())
            os._exit(0)
        os.wait()

thread.start_new_thread(f, ())
thread.start_new_thread(f, ())
thread.start_new_thread(f, ())
thread.start_new_thread(f, ())
thread.start_new_thread(f, ())
f()
------cut here------