[Gcl-devel] RE: comp.lang.lisp posting probs again

[Mike Thomas]

Hi Camm.

| Please accept my apologies and impatience in intruding on your mailbox
| so needlessly.

You are indeed unworthy - just don't allow it to happen again or c.l.l will
be forever lost to you.

| > I didn't touch GCL over the weekend I'm afraid.  I'm trying to set this
| > evening aside for same.
| >
|
| Your efforts are so very much appreciated!

Thanks. I looked further early last week and the TCL/Tk initialisation code
probably needs to be gone over comparing with the relevant files in Tcl/Tk
8.4 as there is a problem initting on Windows.

|  I've just returned from a
| visit to UT/Austin

Sounds like the intro to Little Richard's "Whole Lotta Shakin' Goin' On".


| where there are quite a few (60-100 in the whole
| group, university and industry) heavy GCL users as the predominant
| lisp behind ACL2.  I'll post details to the list, but in short, their
| gratitude was profound and heartfelt.  In their estimation, GCL was
| invaluable, far ahead of the competition.  The level of their
| real-world applications and sophistication was quite impressive --
| hardware design and verification, computational biology, even an
| eventual goal of proof-checking all of SELinux, currently way beyond
| the reach of present capabilities.

That's all great news.

| I had also told them of axiom's
| plans to use acl2 to certify its computer algebra routines, which if
| achieved would place it several steps above other systems -- i.e. the
| main problem with CAS today is that no one knows if they can really
| rely on the answer.

I expect that the problem could be generally computationally undecidable
given that (apparently) mathematics is logically self-inconsistent.  (Not to
take away from the fact that many things might be substantially improved,
but also remembering that even expression simplification is fraught with
difficulty.)


| In any case, this short note is just to consult with you on some
| likely directions GCL will need to take in the future.  The biggest
| one we'll have to work out is parallelism.  The ACL2 people would be
| real users of native threading, and this in my mind gives the task
| some priority.  They have played with openmcl in this regard and have
| a relation with its maintainer, which likely means we can arrange to
| collaborate with him.
|
| What I'm currently thinking is to provide a plet macro that would
| allow fork-based parallelism returning the answer in fasl form across
| a pipe, which I know will be one more mingw incompatibility, but will
| allow us to quickly gain some parallel ability without having to worry
| about gc and reentrancy.  Linux's copy on write pages, combined with a
| trick I have in mind of allocating a bunch of stack memory in the
| child and redirecting alloc_object to use this space first to minimize
| the likelihood of child gc lighting up the copy-on-write, will
| probably be quite performance competitive.


Would MPI help us to achieve this (admittedly less transparently) without
having to worry about source level OS incompatibilities (MPICH works well on
Windows - I once wrote a small subset binding in Haskell)?  My intuition is
that it would be a cost effective approach if it can be manipulated into
doing what you want.

Forking is quite simply a lost cause on Windows.  On the other hand I think
that on OpenMosix distributed systems it is just about the perfect
parallelism solution.


| (In all these considerations, one cannot simply assume 'let forms can
| be executed in parallel as there may be side effects requiring
| sequential evaluation -- the effective lisp model appears to be that
| the user is responsible for invoking the parallelism when they know
| this is not the case, which is trivial in ACL2 as everything is
| functional.  In other words, as best as I can tell, all other memory
| up to the final return value in the child is temporary and can be
| placed on the stack for sizeable performance gains.  openmcl requires
| the user to establish locks around shared globals by hand, and
| specifying the sequential update code to same, but I think a simple
| plet is easier...)
|
| When we figure out threading, we should, I hope, simply be able to
| drop the pipe code and replace the fork machinery with thread calls,
| so I don't think we'll be boxing ourselves in here.
|
| Your thoughts as always are most appreciated.  I'd wager you've had
| more experience with threads than I by far.  I've never found the need
| for them, but have written parallel code in mpi, SYSV shared
| memory/semaphores/message queues, and using fork.


I have only played with threads myself.  Likewise MPI in C and Haskell (I
never finished the GCL MPI work on Windows).  I also once used C* and *Lisp
on a Connection Machine 2 and played a little with OpenMosix.  There is a
*Lisp emulator around on the web which I suppose could be coerced into true
parallelism if some prehistoric front end were thought desirable to be built
on top of your plet idea.

When/if we do threads we should at least consider the portable subset of
pthreads supported on Windows:

  http://sources.redhat.com/pthreads-win32/

although my expectation is that native Windows threads would be faster for
the  relatively simple? functional plet approach you are suggesting.

I am personally interested in GCL parallelism as also support for FEMLisp
(thereby implicitly encouraging/helping the author to enhance same through
parallelism), although I am seriously concerned about my ability to actually
implement these things at the GCL compiler/library level, particularly in
terms of playing catchup with any Unix fork() system.

Cheers

Mike Thomas.