Errors intermixing static and dynamically linked glibc

[Russ Lewis]

Summary:
I have a statically linked executable, which is calling ldopen() to link 
in a library dynamically.  Both the executable and the library call 
printf(); the version from the executable always works, but the one 
called from the library only works under certain conditions.

Details:
Ok, I know I'm doing something really wacky, but here's what I'm 
attempting and the problem I'm facing.  I'm prototyping some changes to 
mmap() for my company, and I need to be able to link some code (along 
with all of the libraries that code uses under the new mmap()). 
Although obviously I have to hack into the kernel, I'm hoping that I 
don't have to hack into glibc as well, if I can avoid it.

So basically, what I'm targeting is writing a loader executable that is 
something like this:

int main()
{
        set_new_mmap_mode_for_this_process();
        ldopen(.....);
        call_entry_point(...);
        return 0;
}

I have to make sure that the library loaded with ldopen gets its own 
copies of all standard libraries, which will be mmap'ed with the new 
feature I'm writing.  Yes, that means that I explicitly want two copies 
of glibc in memory - one linked to the executable, and one linked to the 
library.

The way I've found that (almost) works is to link the loader executable 
statically and link the library dynamically.  When I call ldopen(), 
ldopen doesn't know that glibc symbols are already loaded (via the 
statically linked executable), so it loads a fresh copy of glibc through 
the normal dynamic linking process.  I have confirmed that I am actually 
getting new symbols by doing
        printf("printf = %p\n", printf);
from both the executable and the library.  They have different 
addresses.  A quick dump of /proc/.../maps also shows what I expect.

When I run the loader executable on the command line (without piping it 
into anything), things seem to work well.  Both the executable and the 
library are able to printf() properly, as well as call system() and a 
couple of other functions I've tried.  Good!

However, when I pipe the executable into "| tee log.out", the library's 
version of printf no longer works!  No output.  When I call system() 
(even from the library), the forked process is able to print out to the 
screen just fine...but when I call printf myself, nothing happens. 
fflush() doesn't help.

Some more trivia:

* write(0, ....) always works.
* fileno(stdout) returns 0 from both the executable and the library.
* when I run "strace <loader_exec>", the printf()s called from inside 
the library don't result in write()s.
* If I pass stdout as an argument from the executable to the library, 
then set "stdout = arg" inside the library, then printf starts working 
AFTER the assignment.

So, I assume you're all going to say I'm crazy and that I'm doing 
something that isn't supported.  However, given that, can you guys tell me:
1) Is there a better way to do what I'm attempting?
2) Is there an easy fix to the printf() problem?
3) What other problems am I likely to encounter?  Will malloc() be 
broken?  Other, random, services?  ???

Russ Lewis
        Would-be kernel hacker
        Tucson, Arizona