[Libunwind-devel] libunwind with LD_PRELOAD option

[Shan Shan]

Hi,

  I am trying to implement a customer heap profiler for my ARM9 (arm926ejs) board.

  Basically i have my own malloc, free and load that using LD_PRELOAD before i profile

  the app. I wanted to add stack trace for each alloc/free. I send the LR return address

  from each stack frame to a host where i will do the symbol mapping and make it in 

  human readable format.

  The first part works fine but when i link with libunwind, the app crashes after sending

  some 20K entries. I tried a simple application that does malloc and free with default gcc 

  options and it works fine. This app is compiled with O3 flags for some libraries and

  Os flag for some. basically it is a big beasty app. Here are the details

  libunwind: v1.0-stable

  Toolchain: gcc version 4.3.3 (Sourcery G++ Lite 2009q1-203)

  App: It's a C,C++ mixed app

  Steps: 

  1) Made sure LD_PRELOAD without any unwindings works for my application. It works and stable.

  I track only malloc & free. The code path is mutex protected and ptrs initialized using

  pthread_once. The library is built with the following command

  gcc6 is an alias to Codesourcery compiler.

  gcc6 -D_GNU_SOURCE -fPIC -rdynamic -g -c -Wall <myfiles>;gcc6 -shared -Wl,-soname,librss.so.1 -o librss.so.1.0.1 <my obj files> -lc -ldl -lpthread

  2) Then I built libunwind with Codesourcery tool chain

autoreconf -i

./configure --host=arm-none-linux-gnueabi

  I got the libraries under src/.libs dir.

  3) The i added backtrace inside my code using unwind apis.

    #define UNW_LOCAL_ONLY  // i read it will speed up things with this define

    #include <libunwind.h>

  

    unsigned int mybacktrace (void *buf)

    {

        unsigned int entries = 0;

        unw_cursor_t cursor;

        unw_context_t uc;

        unw_context_t * ctx;

        size_t i = 0;

        int ret;

        if (unw_getcontext(&uc) < 0)

            return 0;

        ctx = &uc;

        if (unw_init_local(&cursor, ctx) < 0)

            return 0;

        do {

            unw_word_t ip;

            unw_word_t sp;

            unw_get_reg(&cursor, UNW_REG_IP, &ip);

            unw_get_reg(&cursor, UNW_REG_SP, &sp);

            buf[entries++] = (void*) ip;

        } while ((ret = unw_step(&cursor)) > 0 && entries < size);

    }

  4) I compiled my library with libunwind like this

  gcc6 -D_GNU_SOURCE -fPIC -rdynamic -g -c -Wall <my input files> -I$LIBUNWIND_DOWNLOAD_PATH/include;gcc6 -shared -Wl,-soname,librss.so.2 -o librss.so.1.0.2 <my obj files> -lc -lpthread -ldl -L$LIBUNWIND_DOWNLOAD_PATH/src/.libs/ -lunwind

  5) I uploaded the new library and libunwind.so.7 to the target. I modified 

  LD_LIBRARY_PATH to add the dir where i have downloaded libunwind.so.7

  6) I started App by setting LD_PRELOAD=$MYPATH/librss.1.0.2, it crashes after sending some

  20K entries. The app is big and have lots of constructors to call at the start. 

  However, this happens only with my library that linked with "libunwind"

  7) To narrow down i wanted to disable stack unwinding atleat during start of app and i

  have a global variable (set to false by default) to check. Now there is no stack trace

  generated during boot but it still crashed. If i remove the -unwind linking, it works

  fine. Is there anything wrong with my compilation?

  I have not touched the app compilation. Basically i want to get this profiling without

  touching the app so that i can run this util on any app any time required. But stuck

  in this issue :-(. 

  Is there anything conceptually wrong to expect this to work?

  I also thought of forcing static linking of -lunwind to my library but i am not sure how. 

  This is a try because of LD_PRELOAD i thought of having most of the symbol lookup

  within the library.

  The core created during crash is not of much use as it could not resolve the symbols

  and points question mark. Besides, the gdb we are using is not good at the moment.

  Help please.

  Thanks,