[Libunwind-devel] UNW_EINVAL stepping past _L_lock_686 in pthread_mutex_lock

[Jared Cantwell]

I apologize if this is a duplicate message.  I sent this earlier, but
I wasn't a member of the list, so I strongly suspect it got silently
dropped on me since I haven't seen it appear in the archive.

I'm working on a tool that sends a signal to all running threads and
records their backtraces using libunwind.  This is useful for getting
a summary of all threads without having to break in with gdb and halt
the entire program.  However, libunwind seems to have trouble walking
the stack of threads that are waiting in pthread_mutex_lock-- unw_step
returns UNW_EINVAL.  I've found similar posts on this list (including
patches), and my version of libunwind is running with those patches.
Is this a known issue?  Is there a way to work around it?

I've included and attached a program that will reproduce the issue
using signals.  I am running with libunwind-1.1.

I am compiling with g++ (4.6.3):
g++ -g unwind_repro.cpp -lpthread -lunwind

Any help or direction is much appreciated.  Let me know if I can
provide more information to help.

~Jared

--------------------------------------------------------------------------------------
#define UNW_LOCAL_ONLY
#include <libunwind.h>

#include <pthread.h>
#include <iostream>
#include <cstdlib>
#include <dlfcn.h>
#include <cxxabi.h>
#include <cassert>

/**
 * This is a test program that reproduces an issue with libunwind
 * unw_step return UNW_EINVAL if the thread is currently waiting on a
 * pthread_mutex_t acquisition.  It gets to __lll_lock_wait and
 * then _L_lock_686, but cannot walk back farther than that.
 *
 * Signals are used to trigger the stack walk while the thread
 * is hung on lock acquisition.
 */

pthread_t new_thread;  // background thread we will signal to get its backtrace
pthread_mutex_t lock;  // lock that will be held when backtrace is attempted

// walk the stack with calls to unw_step
void get_current_stack()
{
    unw_cursor_t cursor;
    unw_context_t uc;
    unw_word_t offp;
    char procname[100];

    unw_getcontext(&uc);
    unw_init_local(&cursor, &uc);

    while(true)
    {
        int res = unw_step(&cursor);

        if (res == 0)
            break;

        // we don't expect this to happen in this test
        if (res < 0)
        {
            std::cout << "FAIL: unw_step returned error; res=" << res
<< std::endl;
            assert(res >= 0);
            break;
        }

        unw_get_proc_name(&cursor, procname, sizeof(procname), &offp);

        int status;
        char *fixedName = abi::__cxa_demangle(procname, NULL, NULL, &status);

        if (fixedName == NULL)
            fixedName = procname;

        std::cout << fixedName << std::endl;

        // Free fixedName if it was allocated
        if (fixedName != procname)
            free(fixedName);
    }
}

// background thread that first gets the current
// stack to make sure the code works, and then
// takes the lock so a signal can be sent to get
// the backtrace while the lock is held.
void* thread_start(void*)
{
    new_thread = pthread_self();

    std::cout << "--- not in pthread_mutex_lock ---" << std::endl;
    get_current_stack();
    std::cout << "---------------------------------" << std::endl;

    pthread_mutex_lock(&lock);
    pthread_mutex_unlock(&lock);
}

// signal handler that is executed while the thread
// above is holding the lock in order to demonstrate
// the issue.
void sig_handler(int signum)
{
    std::cout << "--- IN pthread_mutex_lock ---" << std::endl;
    get_current_stack();
    std::cout << "---------------------------------" << std::endl;
}

int main(int argc, char *argv[])
{
    // initialize and take the global lock so that the background
        // thread hangs on it indefinitely so we can send a signal to it.
    pthread_mutex_init(&lock, NULL);
    pthread_mutex_lock(&lock);

    // register the signal handler that will be invoked by
    // the locked-up thread
    signal(SIGUSR2, sig_handler);

    // launch the background thread and give it a moment to start
    // waiting on the lock acquisition (where it will hang).
    pthread_t thread_id;
    pthread_create(&thread_id, NULL, &thread_start, NULL);
    sleep(2);

    // now send the signal to walk the stack while the lock is held
    pthread_kill(new_thread, SIGUSR2);

    sleep(2);
    pthread_mutex_unlock(&lock);
    pthread_mutex_destroy(&lock);
    pthread_join(thread_id, NULL);

    std::cout << "PASS: Full stack was walked." << std::endl;
}

unwind_repro.cpp
Description: Text Data