gcl-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Gcl-devel] [Maxima-discuss] sys-proclaims.lisp


From: Camm Maguire
Subject: Re: [Gcl-devel] [Maxima-discuss] sys-proclaims.lisp
Date: Fri, 27 Jun 2014 16:30:40 -0400
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (gnu/linux)

Greetings!

Thanks so much for your comments!  My responses below can be summarized
in that the state of the art call I'm aiming for is implemented in
master, including improved foreign functions, but this is not ready for
prime time because the compiler is too slow, and needs some kind of
memoization/cache (partially implemented) prior to release.

Richard Fateman <address@hidden> writes:

> Hi Camm --
>
> I haven't been following this in detail,
> but maybe you are doing what we did years ago with Franz Lisp on VAX..
>
> There was a notion of "file compiling"   so that a call from F to G
> where G was defined
> in the same file as F  was done with a faster linkage.  The assumption
> being that the
> definition of G used by F would not be changed.  (By a definition of G
> in another file, or
> I guess by other means, e.g. tracing.   I forget whether we made some
> escape mechanism.)
>
> There would also be a prolog with a way of calling G from elsewhere -- 
> just not as fast a linkage.
>
> The issue back then was that VAX had a fairly expensive general CALL
> instruction that
> was not need if you knew what you were doing in Lisp.
>

The general idea in gcl is to call through a pointer, which on the first
call goes through an elaborate checking routine, which when passed, sets
the pointer directly to the target for subsequent calls.  This can then
be undone at function redefinition, tracing, or debugging time.  This is
called 'fast-linking'.

To be optimal, we want the call to write as little as possible, which
for a simple function, is just the arguments to stack (or register).
These arguments should either be boxed or unboxed as the function
requests.  In addition, closures need the environment, and vararg
functions need the number of arguments, passed as globals.  Finally,
functions returning multiple values need a place to put them, which
traditionally has been a constant (non-passed global), but in master is
a passed global pointing to stack space in the caller, to take one step
in the direction of thread support.

All of these are set for a generic call, which will proceed (though more
slowly) in all cases, but will be set to fast-linking when possible.  To
get as many of the latter as possible, GCL needs to know the callee
function signatures when writing the call.

Traditionally, gcl has a sys-proclaim mechanism, whereby one pass is
made through all the files, emitting the signature in .fn files,
collecting these into #proclaim statements, and recompiling the whole
thing. 

In master, these signatures, together with the defining file and
function src, and other properties, are stored for each loaded compiled
function.   Hence if source files are compiled and loaded in callee first
order, no second pass is ever required.  #'si::do-recomp is an
experimental facility to detect when written calls have not been optimal
and to automatically recompile the files in question.  This runs into
all kinds of difficulties when things are eval-when (:compile), that
must be run only once, i.e. the pcl bootstrap.


> The big issue for GCL and relevance for Maxima, at least in my view,
> is the inability to
> link to foreign functions  (at all?  or conveniently?)   Or maybe it
> is my ignorance of how to
> do it!
>

Traditionally, GCL has defentry and defCfun, both used heavily by say
xgcl-2.  In master, we have the following:

COMPILER>(do-symbols (s :lib) (print s))

LIB:|libgmp| 
LIB:|libm| 
LIB:|libc| 
NIL

COMPILER>(do-external-symbols (s 'LIB:|libm| ) (print s))

|libm|:|atan| 
|libm|:|ctan| 
|libm|:|csqrt| 
|libm|:|clogf| 
|libm|:|acosh| 
|libm|:|ccosh| 
|libm|:|expf| 
|libm|:|atanhf| 
|libm|:|casin| 
|libm|:|cexpf| 
|libm|:|acosf| 
|libm|:|sqrtf| 
|libm|:|exp| 
|libm|:|atanh| 
|libm|:|ccosf| 
|libm|:|ctanh| 
|libm|:|cosh| 
|libm|:|ccoshf| 
|libm|:|cosf| 
|libm|:|atanf| 
|libm|:|cos| 
|libm|:|cacos| 
|libm|:|tanh| 
|libm|:|ctanf| 
|libm|:|csinhf| 
|libm|:|tanf| 
|libm|:|tan| 
|libm|:|asin| 
|libm|:|sinh| 
|libm|:|csin| 
|libm|:|sinf| 
|libm|:|cabs| 
|libm|:|sin| 
|libm|:|catanhf| 
|libm|:|coshf| 
|libm|:|catanh| 
|libm|:|fabs| 
|libm|:|catanf| 
|libm|:|tanhf| 
|libm|:|acoshf| 
|libm|:|asinh| 
|libm|:|csinh| 
|libm|:|asinhf| 
|libm|:|atan2f| 
|libm|:|asinf| 
|libm|:|sinhf| 
|libm|:|csinf| 
|libm|:|atan2| 
|libm|:|cabsf| 
|libm|:|fabsf| 
|libm|:|casinhf| 
|libm|:|logf| 
|libm|:|casinh| 
|libm|:|clog| 
|libm|:|casinf| 
|libm|:|ctanhf| 
|libm|:|csqrtf| 
|libm|:|log| 
|libm|:|cacoshf| 
|libm|:|cacosh| 
|libm|:|catan| 
|libm|:|cacosf| 
|libm|:|cexp| 
|libm|:|acos| 
|libm|:|sqrt| 
|libm|:|ccos| 
NIL

COMPILER>(defdlfun (:double "cblas_ddot" "libblas.so") :int :double* :int 
:double* :int)

|libblas|:|cblas_ddot|

COMPILER>(do-symbols (s :lib) (print s))

LIB:|libgmp| 
LIB:|libblas| 
LIB:|libm| 
LIB:|libc| 
NIL

COMPILER>(compile **)

;; Compiling /tmp/gazonk_14075_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_14075_0.o.
;; Loading /tmp/gazonk_14075_0.o
start address -T 0x141ddb0 ;; Finished loading /tmp/gazonk_14075_0.o
#<function 00000000019e48e0>
NIL
NIL

COMPILER>(|libblas|:|cblas_ddot| 3 (make-array 3 :element-type 'long-float 
:initial-contents '(1.2 2.3 3.4)) 1 (make-array 3 :element-type 'long-float 
:initial-contents '(2.2 3.3 4.4)) 1)

25.189999999999998

COMPILER>(si::save-system "ff")
(sid_amd64)address@hidden:~/debian/gcl/git/gcl/gcl/unixport$ ./ff
GCL (GNU Common Lisp)  2.7.0 ANSI    May 28 2014 15:46:06
Source License: LGPL(gcl,gmp,pargcl), GPL(unexec,bfd,xgcl)
Binary License:  GPL due to GPL'ed components: (XGCL READLINE UNEXEC)
Modifications of this banner must retain notice of a compatible license
Dedicated to the memory of W. Schelter

Use (help) to get some basic information on how to use GCL.
Temporary directory for compiler files set to /tmp/

>(do-symbols (s :lib) (print s))

LIB:|libgmp| 
LIB:|libblas| 
LIB:|libm| 
LIB:|libc| 
NIL

>(|libblas|:|cblas_ddot| 3 (make-array 3 :element-type 'long-float 
>:initial-contents '(1.2 2.3 3.4)) 1 (make-array 3 :element-type 'long-float 
>:initial-contents '(2.2 3.3 4.4)) 1)

25.189999999999998

>


> There may also be an issue of speed of big integers -- I haven't
> looked at this in detail,
> but the benefits of using GMP  for large enough bignums may be easy to
> benchmark.
> Though the relevance of super-long integers in reality is questionable.

We use GMP exclusively now (for many years).  We dynamically link to the
latest version.  In master, many of the functions are brought forward
via the 'LIB:|libgmp| package.

I have never seen any bignum benchmark that was not memory allocation
limited, by far.  If interested, please see my separate post on
integrating GMP with garbage collection.

We have implemented wrappers to use gmp stably in the presence of gc for
a long time now.  This allows us to use fast relocatable memory for
integers. 

There was some code for allocating GMP integers on the stack, but in my
view it is near impossible to discern situations when you know they
won't escape out of the function.

It seems a better idea is to integrate an integer cache with the gc.
I've tried this in several iterations, and have found no benefit so far,
but have a few other ideas.

>
> (One hack that DOES use super-long integers involves encoding polynomials as
> integers, doing arithmetic, and then decoding.  This makes it possible
> to use whatever
> hacks the GMP implementers used (FFT, etc) to multiply
> polynomials. Maybe faster.)
>
> GMP speed various enormously if one can use tiny cpu-specific
> assembler hacks that
> are keyed to the precise variant on intel chip, so it makes a hash of
> building it in to
> a single binary for "all" wintel machines. I assume the SBCL etc
> people are using some
> relatively slow generic version of GMP, but maybe not?
>
> Anyway, the use of foreign functions is essentially the only reason I
> stray from the GCL
> implementation of Maxima -- and then I generally use the (trial
> version) of Allegro CL
> because I know how to use that foreign function interface, and know
> who to ask for
> help if I get into difficulty.
>
>   Loading mostly stuff like high precision floats, intervals,
> random fortran libraries. (Ray recompiles much stuff into Lisp...)
>   Possibly other people might want parallel processing
> features in GCL?
>

I agree with the idea that if lisp is to be really useful, it must be
able to use precompiled routines written in other languages.  I think
the quicklisp/cl-??? library approach of lisp source libraries compiled
on the fly into the implementation of choice is eventually doomed for
anything serious.

> On another tack -- I don't know if you were reading the Maxima mailing
> list --
> but I thought that Bill Schelter had written a couple of pages of C
> code specific to Maxima
> that would assist in factoring of polynomials  (a null-space
> algorithm).  The only code
> that I've found lying around was C coded implementations of modular
> arithmetic, ie
> coefficient multiplication  (ctimes)   etc   to do   a*b mod c where c
> is a fixnum.
> Have you noticed any other code from Bill that was intended to be a
> c-assist to Maxima?
>

#'ctimes et.al. are specially implemented for maxima support and
included in the GCL source in all versions.  There is also the
non-standard MAKE_UNSPECIAL used somewhere in the maxima source.  These
are all I'm aware of at the moment, but I have not made a thorough search.

Take care,

> Best wishes
>
> Richard Fateman
>
>
> On 6/27/2014 9:40 AM, Camm Maguire wrote:
>> Greetings!
>>
>> Raymond Toy <address@hidden> writes:
>>
>>>>>>>> "Camm" == Camm Maguire <address@hidden> writes:
>>>      Camm> Greetings!  In putting together 2.6.11, I've added automatic
>>>      Camm> 'proclaimed-closure detection to GCL's sys-proclaim.lisp 
>>> generating
>>>      Camm> facility.  I noticed maxima bypasses these gcl routines, 
>>> apparently to
>>>      Camm> keep sys-proclaims.lisp from changing to frequently.  As the 
>>> default gcl
>>>
>>> Yes, that's the reason we bpassed gcl routines. We still use the
>>> internal routines, however, to generate the data, but we don't try to
>>> coalesce them all if the arguments types match.
>>>
>>> What does 'proclaimed-closure detection actually do and how does that
>>> affect sys-proclaim?
>>>
>> It enables fast link calling for closures.  pcl has made use of this
>> property for ages.  2.6.11 will have some support for compiling closures
>> and top-level macrolets, etc., so I modified the sys-proclaims mechanism
>> to detect this and proclaim it too.
>>
>> I have a little facility to detect fast link failure, and the output is
>> now greatly reduced, and the test suite accelerated.  If time permits, I
>> will submit a list of functions maxima intends not to compile (eg. in
>> package :make) to make sure we're not missing anything.  Anonymous calls
>> are still slow linked, until version 2.7 (master).  You can play with
>> 2.6.11pre if desired using git tag cygwin.
>>
>>>      Camm> build process is now to build twice anyway, generating a fresh
>>>      Camm> sys-proclaims.lisp each time, can we not just default to the 
>>> routines in
>>>      Camm> gcl and let my improvements flow through?
>>>
>>> Perhaps things have changed, but sys-proclaim isn't supposed to be
>>> generated on every build. Or perhaps my memory of how this all works
>>> is wrong; I don't build with gcl nearly as often as I used to.
>>>
>> Actually, you are right here -- my apologies.  sys-proclaims.lisp
>> appears to be *generated* by default on every build, but then apparently
>> cached in cvs.
>>
>> So I guess we have two options, 1) modify the default build to build
>> twice automatically, and remove sys-proclaims.lisp from the repository,
>> 2) have me output in the form you prefer, which I would not mind.  I
>> prefer 1) from the point of view that gcl generated compiler output
>> should not be cached in maxima's version control system -- which gcl
>> would one use to commit an update to this file?
>>
>> Take care,
>>
>>> Ray
>>>
>>>
>>> _______________________________________________
>>> Gcl-devel mailing list
>>> address@hidden
>>> https://lists.gnu.org/mailman/listinfo/gcl-devel
>>>
>>>
>>>
>>>
>
>
>
>

-- 
Camm Maguire                                        address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah



reply via email to

[Prev in Thread] Current Thread [Next in Thread]