[Gcl-devel] Re: GCL and integration of MPI (ParGCL)

[Camm Maguire]

Greetings!  OK, tried your stuff, and it words for me.  Perhaps the
quickest way to spon t the discrepancy is for me to post this
transcript.  Please take a look and see where we differ.  thanks again
for your work on this!

=============================================================================
camm@:/tmp$ cat foo.h
static char our_c_fun(p,i)
char *p;
int i;
{ return p[i]; }
camm@:/tmp$ cat hhy1.l
(clines "#include \"foo.h\"")
(defentry joe (string int) (char "our_c_fun"))
camm@:/tmp$ gcl
GCL (GNU Common Lisp)  2.6.6 CLtL1    Jan 18 2005 00:13:38
Source License: LGPL(gcl,gmp), GPL(unexec,bfd)
Binary License:  GPL due to GPL'ed components: (READLINE BFD UNEXEC)
Modifications of this banner must retain notice of a compatible license
Dedicated to the memory of W. Schelter

Use (help) to get some basic information on how to use GCL.

>(compile-file "hhy1.l" :system-p t)

Compiling hhy1.l.
End of Pass 1.  
End of Pass 2.  
OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3
Finished compiling hhy1.l.
#p"hhy1.o"

>(compiler::link (list "hhy1.o") "new_image")
GCL (GNU Common Lisp)  April 1994  131072 pages
Building symbol table for /tmp/raw_new_image ..
loading /usr/lib/gcl-2.6.6/unixport/../lsp/gcl_export.lsp
Initializing gcl_defmacro.o
Initializing gcl_evalmacros.o
Initializing gcl_top.o
Initializing gcl_module.o
loading /usr/lib/gcl-2.6.6/unixport/../lsp/gcl_autoload.lsp

>
("hhy1")

>
#<"COMPILER" package>

>
#<"SLOOP" package>

>
#<"SERROR" package>

>
#<"ANSI-LOOP" package>

>
#<"DEFPACKAGE" package>

>
#<"TK" package>

>
#<"SYSTEM" package>

SYSTEM>
*COMMAND-ARGS*

SYSTEM>Initializing gcl_predlib.o
Initializing gcl_setf.o
Initializing gcl_arraylib.o
Initializing gcl_assert.o
Initializing gcl_defstruct.o
Initializing gcl_describe.o
Initializing gcl_iolib.o
Initializing gcl_listlib.o
Initializing gcl_mislib.o
Initializing gcl_numlib.o
Initializing gcl_packlib.o
Initializing gcl_seq.o
Initializing gcl_seqlib.o
Initializing gcl_trace.o
Initializing gcl_sloop.o
Initializing gcl_serror.o
Initializing gcl_destructuring_bind.o
Initializing gcl_loop.o
Initializing gcl_defpackage.o
Initializing gcl_make_defpackage.o
Initializing gcl_cmpinline.o
Initializing gcl_cmputil.o
Initializing gcl_debug.o
Initializing gcl_info.o
Initializing gcl_cmptype.o
Initializing gcl_cmpbind.o
Initializing gcl_cmpblock.o
Initializing gcl_cmpcall.o
Initializing gcl_cmpcatch.o
Initializing gcl_cmpenv.o
Initializing gcl_cmpeval.o
Initializing gcl_cmpflet.o
Initializing gcl_cmpfun.o
Initializing gcl_cmpif.o
Initializing gcl_cmplabel.o
Initializing gcl_cmplam.o
Initializing gcl_cmplet.o
Initializing gcl_cmploc.o
Initializing gcl_cmpmap.o
Initializing gcl_cmpmulti.o
Initializing gcl_cmpspecial.o
Initializing gcl_cmptag.o
Initializing gcl_cmptop.o
Initializing gcl_cmpvar.o
Initializing gcl_cmpvs.o
Initializing gcl_cmpwt.o

Loading /usr/lib/gcl-2.6.6/unixport/../lsp/sys-proclaim.lisp
Finished loading /usr/lib/gcl-2.6.6/unixport/../lsp/sys-proclaim.lisp
Loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/sys-proclaim.lisp
Finished loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/sys-proclaim.lisp
Loading /usr/lib/gcl-2.6.6/unixport/../gcl-tk/tk-package.lsp
Finished loading /usr/lib/gcl-2.6.6/unixport/../gcl-tk/tk-package.lsp
Loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/gcl_cmpmain.lsp
Warning: COMPILE-FILE is being redefined.
Warning: COMPILE is being redefined.
Warning: DISASSEMBLE is being redefined.
Finished loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/gcl_cmpmain.lsp
Loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/gcl_lfun_list.lsp
Finished loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/gcl_lfun_list.lsp
Loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/gcl_cmpopt.lsp
Finished loading /usr/lib/gcl-2.6.6/unixport/../cmpnew/gcl_cmpopt.lsp
Loading /usr/lib/gcl-2.6.6/unixport/../lsp/gcl_auto_new.lsp
Finished loading /usr/lib/gcl-2.6.6/unixport/../lsp/gcl_auto_new.lsp

Initializing hhy1.o
T

>
"new_image"

>(by)
camm@:/tmp$ ./new_image
GCL (GNU Common Lisp)  2.6.6 CLtL1    Jan 18 2005 00:13:38
Source License: LGPL(gcl,gmp), GPL(unexec,bfd)
Binary License:  GPL due to GPL'ed components: (READLINE BFD UNEXEC)
Modifications of this banner must retain notice of a compatible license
Dedicated to the memory of W. Schelter

Use (help) to get some basic information on how to use GCL.

>(joe "abcd" 3)

#\d

>(by)
=============================================================================

Take care,

Gene Cooperman <address@hidden> writes:

> Thanks for the quick reply.  I have some time right now, and I would
> be happy to set up pargcl as a configurable option in the distribution.
> Of course, before doing that, I should resuscitate it for gcl-2.6.6.
> 
> If compiler::link is the preferred way to link in foreign functions
> right now, I'm happy to set it up that way.
> 
> I'm attaching two files:
>   foreign-fnc.lsp
>   foreign-fnc.ch
> 
> To use them, I copy everything into gcl-2.6.6/unixport for
> convenience.  I then execute 'gcl', and:
>   (compile-file "foreign-fnc.lsp" :system-p t)
>   (load "foreign-fnc.o") ; for testing, this works
>   (joe "abcdefg" 2) ; for testing, this works
>   ;; Next, save directly into unixport, so the gcl script will pick it up next
>   ;;   time.  As you say, the fifth argument defaults to t, as desired.
>   (compiler::link (list "foreign-fnc.o") "saved_gcl")
>   (quit)
> Then I again execute 'gcl':
>   (apropos "joe")  ; nothing returned
>   (joe "abcdefg" 2) ; error returned, no such function
> 
> On my home computer (Debian testing, x86), I get the above.
> I just tried at my work computer,  I saved to saved_gcl2, and
> then did:  cp saved_gcl2 saved_gcl
> to avoid busy error.  I then execute 'gcl', and now get:
> >(apropos "joe")
> COMPILER::JOE
> 
> >(compiler::joe "abcdefg" 2)
> 
> Error: The function COMPILER::JOE is undefined.
> ...
> 
> >(joe "abcdefg" 2)
> 
> Error: The function JOE is undefined.
> 
> 
> 
> 
> 
> Thanks for the help.          - Gene
> 
> On Wed, Jun 15, 2005 at 11:27:52AM -0400, Camm Maguire wrote:
> > Greetings!
> > 
> > It would be nice to have pargcl as a configurable option in the
> > standard gcl distribution.
> > 
> > Gene Cooperman <address@hidden> writes:
> > 
> > > I noticed recently that with gcl-2.6.x, my ParGCL distribution doesn't
> > > seem to work any more.  The issue is that I need to statically
> > > link a file (mpi_glue.o) and a static library (libmpi.a).
> > > In the past, I could do this with:
> > >   rm -f raw_saved_gcl
> > >   make EXTRAS="mpi_glue.o libmpi.a"
> > > 
> > > It appears that this no longer works.  If I read the documentation
> > > correctly, it's because gcl is now using bfd for linking.  (Is this 
> > > right?)
> > > 
> > 
> > I thought this still worked.  It would be great if you could post the
> > symptoms of the failure.  It is true that I believe this method is
> > obsolete, perhaps even 'deprecated' :-).
> > 
> > > The documentation also seems to say that I need to now use
> > >  compiler::link .  I tried this, and while compiler::link successfully
> > > creates a new saved_gcl, my foreign functions are not available in
> > > the new saved_gcl.
> > > 
> > 
> > OK, assuming mpi_glue.o was output by (compile-file ... :system-p t),
> > I think you need
> > 
> > (compiler::link (list "mpi_glue.o") "new_image" "" "libmpi.a" t)
> > 
> > First arg -- list of compiled lisp modules with :system-p t flag.
> > Second -- output image name
> > Third  -- Any interpreted initialization code needed to be run in
> >           new_image
> > Fourth -- C libs and objects not produced with GCL's compiler,
> >           including dynamic libs (i.e. could just do -lmpi)
> > Fifth  -- whether to initialize the lisp modules in the first arg
> >           explicitly, or to do so by way of redirecting calls to load
> >           invoked in the third argument.  I.e. in acl2, we just run
> >           their massive init sequence again, and all calls to load of 
> > modules
> >           already linked in simply run the initialization sequence.  In all
> >           cases, this init process needs doing at some point for the 
> > functions
> >           in the module to become accessible.  You can crudely think of it 
> > as
> >           allocating the function structure bodies and naming symbols and
> >           pointing them at the right code address in the new_image.
> > 
> > If mpi_glue.o is pure C, then it belongs in the 4rth arg instead.
> > 
> > > As a test, I try to make a simple file, "foreign-fnc.lsp", according
> > > to the example in the "defentry" section of the GCL manual.
> > > I can create it, compile it, load it, and use it successfully.
> > > 
> > > Then I try:
> > >   cd gcl-2.6.6/unixport; gcl
> > > and:
> > >   (compile-file "foreign-fnc.lsp" :system-p t)
> > >   (compiler::link '("foreign-fnc.o") "saved_gcl")
> > > I see:
> > > Initializing foreign-fnc.o
> > > T
> > > >
> > > "saved_gcl"
> > > 
> > > It creates a new saved_gcl, but the new one does not contain the
> > > foreign function that I defined.
> > > 
> > 
> > OK, the fifth arg defaulted to t, so all should be initialized.  I
> > don't know why this did not work -- could you please post your .lsp
> > file?
> > 
> > 
> > > I've also tried:
> > >   (compiler::link '("foreign-fnc.o") "saved_gcl" "(load 
> > > \"foreign-fnc.o\")")
> > > but it seems to have similar results.
> > 
> > I take it you are working on a systm with native relocation abilities,
> > such as x86.  In this case, there is never a need for use of the first
> > argument, as GCL can just (load "foo.o") followed by (si::save-system
> > "foo") to the same effect.  You can, but do not have to, link in
> > compiled lisp modules with compiler::link.  YOu only absolutely need
> > it for pure C code and libraries.  I would just
> > 
> > (compiler::link nil "new_image" "" "-lmpi my_C_module.o")
> > 
> > fire up new_image, and load your lisp interface from there.  If this
> > works, you can then do all in one step via compiler::link's third arg.
> > 
> > If you point me at your files again, I might find some time to test too.
> > 
> > Take care,
> > 
> > > 
> > > Do you have any suggestions?
> > >                                                   Thanks,
> > >                                                   - Gene
> > > 
> > > 
> > > 
> > 
> > -- 
> > Camm Maguire                                                address@hidden
> > ==========================================================================
> > "The earth is but one country, and mankind its citizens."  --  Baha'u'llah
> 
> ;;; From info/gcl-si.info
> 
> ;; JOE takes X a lisp string and Y a fixnum and returns a character.
> (clines "#include \"foreign-fnc.ch\"")
> (defentry joe (string int) (char "our_c_fun"))
> 
> ;;;; In foreign-fnc.ch :
> ;;; /* C function for extracting the i'th element of a string */
> ;;; static char our_c_fun(p,i)
> ;;; char *p;
> ;;; int i;
> ;;; { return p[i]; }
> 
> /* C function for extracting the i'th element of a string */
> static char our_c_fun(p,i)
> char *p;
> int i;
> { return p[i]; }

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah