Re: [Tinycc-devel] tcc_relocate() and tcc_relocate_ex()

[Sean Conner]

It was thus said that the Great grischka once stated:
> Sean Conner wrote:
> >  Can you at least commit the change that makes tcc_relocate_ex() global?
> 
> Yes, perhaps at least.  You wrote:
> >... but fixing my code to match the current definition
> >of tcc_relocate() was out of the question (the surrounding code manages the
> >lifetime of both the TCCState and the resulting executable code compiled
> >into memory).
> 
> I'd actually like to know some details why you can't just use 
> tcc_relocate()?
> Admittedly we know little about how people actually use the libtcc API.

  I have three use cases right now in using libtcc.  

Case 1:  Synthesis OS (http://valerieaurora.org/synthesis/SynthesisOS/)

  Synthesis OS was an experiment operating system from 1992 written for a
Ph. D. dissertation.  Dispite it being written in assembly language
(MC68030) it basically would "recompile" system calls on the fly (the whole
thing is mind blowing and its worth reading the dissertation)---think of a
"just-in-time" compiler but one that recompiles the kernel as needed.  This
is a technique I'm interested in (on my own time).

  I have a "wow---it can work!" type of prototype using libtcc (nothing even
close to a kernel, just some simple code).  I have a custom "open()"
function that will compile a custom "read()" routine based upon the
paramteres given it---if I open a file read-only for character-by-character
access, and if the file can be mmap()'ed into memory, then you get a
specialized version of read() that returns successive characters from an
mmap()'ed file; if the file can't be mmap()'ed, you get a more traditional
read() function.  This re-compilation happens on a file-by-file basis (each
file returned from my custom open() will have a custom read() function
compiled that deals only with the given file).

  The issue here with tcc_relocate() is that once the compilation phase is
done, I don't need (or want) the symbol table, section definitions, buffers,
or anything else of the TCCState *except for the code just compiled*, and
the memory for that (the compiled code) is reclaimed on the call to a custom
close() function.

  Yes, I could skip the compilation step and load pre-written functions (as
a shared object) but the ability to compile a custom version, quickly (one
of the strengths of TCC) in memory and use it immediately, is not something
I want to give up here.

Case 2:  Lua (http://www.lua.org/)

  Just to recap Lua---it's a small scripting language intended to be
embedded in a larger application to give said application scripting
abilities.  You can also load additional modules either written in Lua or
written in C (as shared objects, either .so for Unix or .dll for Windows).  

  Anyway, I wrote Lua bindings for libtcc [1], so from within a Lua script,
I can include C code, compile it and then call the newly compiled C code
from Lua (you can see an example of this starting with line 112 of [2]). 
Just as with case 1, once the code is compiled, I don't care about the rest
of the TCCState---just the compiled code.

  In fact, I use the libtcc bindings to extend Lua's require() function to
compile a Lua module upon loading it, and have Lua manage the memory for the
resulting code automatically (Lua is a garbage collected language).

        [1] https://github.com/spc476/lua-conmanorg/blob/master/src/tcc.c
        [2] https://github.com/spc476/lua-conmanorg/blob/master/lua/cc.lua

Case 3: Closures in C (or, a mixture of case 1 and case 2)

  Why this exists doesn't matter, but I have written Lua bindings to Xlib
(the lowest level of X programming without talking the protocol directly). 
Xlib, however, reports some errors via a callback function you register
using XSetErrorHandler():

        int (*XSetErrorHandler(int (*handler)(Display *, XErrorEvent *)))();

  I would like to push the error into Lua, but the issue I have is---I need
a global variable pointing to the current Lua state, which is something I
don't particularly want (the global variable), because the following:

        int foo(lua_State *L)
        {
          XSetErrorHandler(int (Display *display,XEerrorEvent *event)
                {
                  lua_pushlightuserdata(L,display);
                  lua_gettable(L,LUA_REGISTRYINDEX);
                  lua_getfield(L,-1,"error_handler");
                  mylua_pushevent(L,event);
                  lua_call(L,1,0);
                  return 0;
                }
          )
          return 0;
        }

  is not legal C.  However, using libtcc, I can fake a closure with C:

        #include <stdlib.h>
        #include <X11/Xlib.h>
        #include <syslog.h>
        #include <lua.h>
        #include <lualib.h>
        #include <lauxlib.h>

        lua_State *L;

        int Xcallback_function(Display *display,XErrorEvent *event)
        {
          lua_pushlightuserdata(...);
          /* ... */
        }

and compile it with (excluding error checking for clarity):

        lua_State  *L; /* assume it's set properly */
        TCCState   *s;
        int         size;
        void       *mem;
        lua_State **pL;
        int       (*cb)(Display *,XErrorEvent *);

        s = tcc_new();
        tcc_set_output_type(s,TCC_OUT_MEMORY);
        tcc_add_file(s,"xcallback.c");
        size = tcc_relocate_ex(s,NULL);
        mem = lua_newuserdata(L,size);
        tcc_relocate_ex(s,mem);
        pL = tcc_get_symbol(s,"L");
        cb = tcc_get_symbol(s,"Xcallback_function");
        *pL = L;
        XSetErrorHandler(cb);

        /*----------------------------------------
        ; I don't need the compiler info any more
        ;----------------------------------------*/

        tcc_delete(s);

  Yes, it's a bit of a ways to go to get closures under C, but it works.

  And in all cases, once the code is compiled, I have no need for the
compiler specific stuff (symbols, section definitions, etc), but I want the
code.  In order to get that, I need tcc_relocate_ex()---tcc_relocate() does
a bit *too* much for me, and leaves around a ton of stuff I don't care about
after compiling the code.  

  In a way, I'm embedding libtcc into an application to allow me to extend
the application with C (much like you can do with Lua).  

  -spc (So, I hope that answers your question ... )