[Gcl-devel] New dynamic library access in GCL

[Camm Maguire]

Greetings!  GCL now has the ability to access arbitrary external
shared library routines in a persitent fashion -- i.e. the binding is
kept across image saves:

=============================================================================
address@hidden:/fix/t1/camm/debian/gcl/tmp/tmp/foo1$ unixport/saved_gcl
GCL (GNU Common Lisp)  2.7.0 CLtL1    Jun 15 2007 19:45:46
Source License: LGPL(gcl,gmp,pargcl), GPL(unexec,bfd,xgcl)
Binary License:  GPL due to GPL'ed components: (XGCL READLINE BFD UNEXEC)
Modifications of this banner must retain notice of a compatible license
Dedicated to the memory of W. Schelter

Use (help) to get some basic information on how to use GCL.

Temporary directory for compiler files set to /tmp/

>(|libm|:|erf| 1.0)

Error: ERROR "Cannot find the external symbol erf in #<\"libm\" package>."
Fast links are on: do (si::use-fast-links nil) for debugging
Signalled by READ.
ERROR "Cannot find the external symbol erf in #<\"libm\" package>."

Broken at READ.  Type :H for Help.
>:q

>(si::show-lib-syms)

(LIB:|libm| 1074447640 #<"libm" package>) 
(|libm|:|atan| 1094141472 #<compiled-function |libm|:|atan|>) 
(|libm|:|ctan| 1094156208 #<compiled-function |libm|:|ctan|>) 
(|libm|:|csqrt| 1094159152 #<compiled-function |libm|:|csqrt|>) 
(|libm|:|clogf| 1094183232 #<compiled-function |libm|:|clogf|>) 
(|libm|:|acosh| 1094146080 #<compiled-function |libm|:|acosh|>) 
(|libm|:|ccosh| 1094153584 #<compiled-function |libm|:|ccosh|>) 
(|libm|:|expf| 1094176432 #<compiled-function |libm|:|expf|>) 
(|libm|:|atanhf| 1094176064 #<compiled-function |libm|:|atanhf|>) 
(|libm|:|casin| 1094154864 #<compiled-function |libm|:|casin|>) 
(|libm|:|cexpf| 1094181648 #<compiled-function |libm|:|cexpf|>) 
(|libm|:|acosf| 1094175504 #<compiled-function |libm|:|acosf|>) 
(|libm|:|sqrtf| 1094180496 #<compiled-function |libm|:|sqrtf|>) 
(|libm|:|exp| 1094146864 #<compiled-function |libm|:|exp|>) 
(|libm|:|atanh| 1094146512 #<compiled-function |libm|:|atanh|>) 
(|libm|:|ccosf| 1094184128 #<compiled-function |libm|:|ccosf|>) 
(|libm|:|ctanh| 1094156960 #<compiled-function |libm|:|ctanh|>) 
(|libm|:|cosh| 1094146672 #<compiled-function |libm|:|cosh|>) 
(|libm|:|ccoshf| 1094182688 #<compiled-function |libm|:|ccoshf|>) 
(|libm|:|cosf| 1094172320 #<compiled-function |libm|:|cosf|>) 
(|libm|:|atanf| 1094172032 #<compiled-function |libm|:|atanf|>) 
(|libm|:|cos| 1094141792 #<compiled-function |libm|:|cos|>) 
(|libm|:|cacos| 1094157584 #<compiled-function |libm|:|cacos|>) 
(|libm|:|tanh| 1094145584 #<compiled-function |libm|:|tanh|>) 
(|libm|:|ctanf| 1094185024 #<compiled-function |libm|:|ctanf|>) 
(|libm|:|csinhf| 1094182144 #<compiled-function |libm|:|csinhf|>) 
(|libm|:|tanf| 1094175168 #<compiled-function |libm|:|tanf|>) 
(|libm|:|tan| 1094145536 #<compiled-function |libm|:|tan|>) 
(|libm|:|asin| 1094146208 #<compiled-function |libm|:|asin|>) 
(|libm|:|sinh| 1094150832 #<compiled-function |libm|:|sinh|>) 
(|libm|:|csin| 1094155520 #<compiled-function |libm|:|csin|>) 
(|libm|:|sinf| 1094175120 #<compiled-function |libm|:|sinf|>) 
(|libm|:|cabs| 1094152304 #<compiled-function |libm|:|cabs|>) 
(|libm|:|sin| 1094145488 #<compiled-function |libm|:|sin|>) 
(|libm|:|catanhf| 1094187072 #<compiled-function |libm|:|catanhf|>) 
(|libm|:|coshf| 1094176224 #<compiled-function |libm|:|coshf|>) 
(|libm|:|catanh| 1094158672 #<compiled-function |libm|:|catanh|>) 
(|libm|:|fabs| 1094144096 #<compiled-function |libm|:|fabs|>) 
(|libm|:|catanf| 1094183504 #<compiled-function |libm|:|catanf|>) 
(|libm|:|tanhf| 1094175216 #<compiled-function |libm|:|tanhf|>) 
(|libm|:|acoshf| 1094175632 #<compiled-function |libm|:|acoshf|>) 
(|libm|:|asinh| 1094141232 #<compiled-function |libm|:|asinh|>) 
(|libm|:|csinh| 1094152928 #<compiled-function |libm|:|csinh|>) 
(|libm|:|asinhf| 1094171792 #<compiled-function |libm|:|asinhf|>) 
(|libm|:|atan2f| 1094175888 #<compiled-function |libm|:|atan2f|>) 
(|libm|:|asinf| 1094175760 #<compiled-function |libm|:|asinf|>) 
(|libm|:|sinhf| 1094180368 #<compiled-function |libm|:|sinhf|>) 
(|libm|:|atan2| 1094146336 #<compiled-function |libm|:|atan2|>) 
(|libm|:|csinf| 1094184448 #<compiled-function |libm|:|csinf|>) 
(|libm|:|cabsf| 1094181552 #<compiled-function |libm|:|cabsf|>) 
(|libm|:|fabsf| 1094174288 #<compiled-function |libm|:|fabsf|>) 
(|libm|:|casinhf| 1094186160 #<compiled-function |libm|:|casinhf|>) 
(|libm|:|logf| 1094179104 #<compiled-function |libm|:|logf|>) 
(|libm|:|casinh| 1094157680 #<compiled-function |libm|:|casinh|>) 
(|libm|:|clog| 1094154112 #<compiled-function |libm|:|clog|>) 
(|libm|:|casinf| 1094183888 #<compiled-function |libm|:|casinf|>) 
(|libm|:|ctanhf| 1094185616 #<compiled-function |libm|:|ctanhf|>) 
(|libm|:|csqrtf| 1094187472 #<compiled-function |libm|:|csqrtf|>) 
(|libm|:|log| 1094149536 #<compiled-function |libm|:|log|>) 
(|libm|:|cacoshf| 1094186624 #<compiled-function |libm|:|cacoshf|>) 
(|libm|:|cacosh| 1094158144 #<compiled-function |libm|:|cacosh|>) 
(|libm|:|catan| 1094154432 #<compiled-function |libm|:|catan|>) 
(|libm|:|cacosf| 1094186080 #<compiled-function |libm|:|cacosf|>) 
(|libm|:|cexp| 1094152400 #<compiled-function |libm|:|cexp|>) 
(|libm|:|acos| 1094145952 #<compiled-function |libm|:|acos|>) 
(|libm|:|sqrt| 1094150976 #<compiled-function |libm|:|sqrt|>) 
(|libm|:|ccos| 1094155168 #<compiled-function |libm|:|ccos|>) 
(|libm|:|abs| 1090798240 #<compiled-function |libm|:|abs|>) 
(LIB:|libc| 1074448272 #<"libc" package>) 
(|libc|:|setjmp| 1090786864 NIL) 
(|libc|:|feof| 1090993296 NIL) 
(|libc|:|memset| 1091054096 NIL) 
(|libc|:|getc| 1090994720 NIL) 
(|libc|:|bzero| 1091054432 NIL) 
(|libc|:|putc| 1090995664 NIL) 

>(in-package 'compiler)

#<"COMPILER" package>

COMPILER>(defdlfun (:double "erf" "libm.so") :double)

|libm|:|erf|

COMPILER>(compile *)

;; Compiling /tmp/gazonk_18613_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_18613_0.o.
;; Loading /tmp/gazonk_18613_0.o
 ;; start address -T 0xa02000 ;; Finished loading /tmp/gazonk_18613_0.o
#<compiled-function |libm|:|erf|>
NIL
NIL

COMPILER>(|libm|:|erf| 1.0)

0.84270079294971489

COMPILER>(|libm|:|erf| 1.0s0)

Correctable error: TYPE-ERROR :DATUM 1.0S0 :EXPECTED-TYPE LONG-FLOAT NIL
Fast links are on: do (si::use-fast-links nil) for debugging
Signalled by EVAL.
If continued: choose a new value
TYPE-ERROR :DATUM 1.0S0 :EXPECTED-TYPE LONG-FLOAT NIL

Broken at EVAL.  Type :H for Help.
COMPILER>>:q

Top level.
COMPILER>(disassemble '(lambda (x) (|libm|:|erf| x)) nil)

;; Compiling /tmp/gazonk_18613_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_18613_0.o.

#include "gazonk_18613_0.h"
void init_code(){do_init((void *)VV);}
/*      local entry for function CMP-ANON       */

static object LI1(V2)

object V2;
{        VMB1 VMS1 VMV1
        goto TTL;
TTL:;
        {object V3 = (/* erf */(*LnkLI0)((V2)));VMR1
        (V3);}
        return Cnil;
}
static object  LnkTLI0(object first,...){object V1;va_list 
ap;va_start(ap,first);V1=(object )call_proc_new(((object)VV[0]),0,0,(void 
**)(void *)&LnkLI0,1,first,ap);va_end(ap);return V1;} /* erf */
#(#(erf
    (%INIT
     . #((LET ((*DISABLE-RECOMPILE* T))
           (MFSFUN 'CMP-ANON 0 1 0)
           (ADD-HASH 'CMP-ANON '((T) T) '((erf (FLOAT) T))COMPILER
libmerf-
               '/tmp/gazonk_18613_0.lsp))
         (DO-RECOMPILE)))))
static object LI1();
#define VMB1
#define VMS1
#define VMV1
#define VMR1(VMT1) return(VMT1);
#define VM1 0
static void * VVi[2]={
#define Cdata VV[1]
(void *)(LI1)
};
#define VV (VVi)
static object  LnkTLI0(object,...);
static object  (*LnkLI0)() = (object (*)()) LnkTLI0;
NIL

COMPILER>(disassemble '(lambda (x) (declare (long-float x)) (|libm|:|erf| x)) 
nil)

;; Compiling /tmp/gazonk_18613_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_18613_0.o.

#include "gazonk_18613_0.h"
void init_code(){do_init((void *)VV);}
/*      local entry for function CMP-ANON       */

static double LI1(V2)

double V2;
{        VMB1 VMS1 VMV1
        goto TTL;
TTL:;
        {double V3 = ((double(*)(double))dlerf)(V2);VMR1
        (V3);}
}
/*      global entry for the function CMP-ANON  */

static void L1()
{       register object *base=vs_base;
        base[0]=make_longfloat(LI1(lf(base[0])));
        vs_top=(vs_base=base)+1;
}
#(#(NIL
    (%INIT
     . #((MDL 'erf 'libm 1)
         (LET ((*DISABLE-RECOMPILE* T))
           (MF 'CMP-ANON 0)
           (ADD-HASH 'CMP-ANON '((LONG-FLOAT) LONG-FLOAT)
               '((erf (FLOAT) T))
LISPLAMBDA!
!,DECLAR,OPTIMIZ,SAFETY
libmerf-
               '/tmp/gazonk_18613_0.lsp))
         (DO-RECOMPILE)))))
static void L1();
static double LI1();
static void *dlerf;
#define VMB1
#define VMS1
#define VMV1
#define VMR1(VMT1) return(VMT1);
#define VM1 0
static void * VVi[2]={
#define Cdata VV[1]
(void *)(L1),
(void *)(&dlerf)
};
#define VV (VVi)
NIL

COMPILER>(funcall (compile nil  '(lambda (x) (|libm|:|erf| x))) 1.0)

;; Compiling /tmp/gazonk_18613_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_18613_0.o.
;; Loading /tmp/gazonk_18613_0.o
 ;; start address -T 0x9e8b08 ;; Finished loading /tmp/gazonk_18613_0.o
0.84270079294971489

COMPILER>(funcall (compile nil  '(lambda (x) (declare (long-float x)) 
(|libm|:|erf| x))) 1.0)

;; Compiling /tmp/gazonk_18613_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_18613_0.o.
;; Loading /tmp/gazonk_18613_0.o
 ;; start address -T 0xa08948 ;; Finished loading /tmp/gazonk_18613_0.o
0.84270079294971489

COMPILER>(si::save-system "/tmp/h")
address@hidden:/fix/t1/camm/debian/gcl/tmp/tmp/foo1$ /tmp/h
GCL (GNU Common Lisp)  2.7.0 CLtL1    Jun 15 2007 19:45:46
Source License: LGPL(gcl,gmp,pargcl), GPL(unexec,bfd,xgcl)
Binary License:  GPL due to GPL'ed components: (XGCL READLINE BFD UNEXEC)
Modifications of this banner must retain notice of a compatible license
Dedicated to the memory of W. Schelter

Use (help) to get some basic information on how to use GCL.

Temporary directory for compiler files set to /tmp/

>(|libm|:|erf| 1.0)

0.84270079294971489

>
=============================================================================

Notes: 

0) based on dlopen
1) Not yet tested on static linking
2) Cannot run such functions interpreted for the moment
3) compiling gives both a function with error checking, and an inline
   providing single instuction access through a C pointer where
   possible.
4) Plan on shipping a little blas, maybe mpi and lapack file to be
   optionally loaded in the GCL distribution
5) package LIB contains libary name symbols bound to the dlopen
   address of the library
6) each library has its own package with symbols bound to the external
   function address.
7) symbols are created and linked on .o load if necessary
8) loaded .o code keeps a list of its external pointers in use, which
   are then reset on image re-execution.

Enjoy!

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah