Re: [Libunwind-devel] libunwind with LD_PRELOAD option

[Lassi Tuura]

Hi,

>   I am trying to implement a customer heap profiler for my ARM9 (arm926ejs) 
> board.
>   Basically i have my own malloc, free and load that using LD_PRELOAD before 
> i profile
>   the app. I wanted to add stack trace for each alloc/free. I send the LR 
> return address
>   from each stack frame to a host where i will do the symbol mapping and make 
> it in 
>   human readable format.
> 
>   The first part works fine but when i link with libunwind, the app crashes 
> after sending
>   some 20K entries. I tried a simple application that does malloc and free 
> with default gcc 
>   options and it works fine. This app is compiled with O3 flags for some 
> libraries and
>   Os flag for some. basically it is a big beasty app. Here are the details

You don't mention where the sending stacks to other machine comes in your
workflow and exact information about the crash causes. If you are writing
'malloc' profiler, you need to be very careful what you do while inside the
'malloc' call. If you are sending stacks elsewhere in the callback itself,
you have to be extremely careful not to trigger for example recursive calls
to malloc. You will also get calls from various bits of your libc, so you
need to be certain you do not do anything which will destabilise those. Do
beware all sorts of places call 'malloc', some you may find unexpected.

The unwind code you posted seems reasonable. I don't know ARM implementation
of libunwind, but I assume it doesn't make hazardous malloc calls; on x86-64
your implementation should be safe. I'd check the code which calls this first.

I've written one malloc profiler (called igprof), and we regularly profile
apps which allocate at 1-3 MHz rate, consist of hundreds of shared libraries
and run for hours, so I can offer advice on some practical aspects. I know
little about ARM tool chain however.

Regards,
Lassi