gcl-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Gcl-devel] Re: Memory thrashing


From: Camm Maguire
Subject: [Gcl-devel] Re: Memory thrashing
Date: 30 Dec 2005 18:01:27 -0500
User-agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2

Greetings!  Great to see your progress!!  Awesome!

"Mike Thomas" <address@hidden> writes:

> Hi Camm.
> 
> Please find below output from (room) on two newly started, completely
> unworked unixport/saved_ansi_gcl.exe images.  One is GCL HEAD,  the other is
> 2.6.8pre.
> 
> I'm posting these because the GCL HEAD random tester left running overnight
> is about three times faster this morning than when I left it last night.  It
> just did 600 iterations in half an hour compared with about 400 in one whole
> hour last night.
> 

Yes, you have found two changes in cvs HEAD leading to compiler
slowness.  I believe this will be temporary once the compiler
type-propagation algorithm is finalized, but in any case, it currently
serves as a price for *much improved* execution speed of compiled code
in general.  You might want to skim the messages between Bob Boyer and
I as posted to the list on this.

Regarding memory, I'd be shocked if you were not equilibrating faster
than an entire hour.  Perhaps it is work checking room at certain
intervals, e.g. interrupting with Control-C and resuming.  We have
scaled now (I believe) all memory size parameters with the size of the
core.  Most notably, the hole is a fraction of the core size, and this
does (very modestly) slow down certain memory operations as they try
to rapidly expand the heap.  It was felt that this was a good tradeoff
to support compact applications.  In any case, you can use
(si::set-hole-size 13107)(si::gbc t) should you desire.

Secondly, the lion's share of the compile time slowdown is in the
current algorithm to autodeclare certain let bindings.  At present, it
is exponential in the let nesting, truncated at a certain depth.  If
you want fast compiles just to get testing things more efficiently,
you can do

(setq compiler::*dlbsrl* 0) ;; i.e. declare let bindings recursion limit

This should give you constant let binding declarations, but miss any
possible auto declarations for changed let bindings, which in any case
aren't in cvs yet anyway.

Even without this, building the ansi image only takes about 15min on
my somewhat older machine anyway.  The random tester, of course, will
be noticeably slower.

I've been mulling over a decent algorithm.  I think we can get one
that is merely iterative on tagbodies, and O(1) otherwise.  It goes
like this (feedback most welcome!):

Expand the var struct to hold two type stacks per variable, "current"
and "maximal".  Assign the type of the binding form to the first
element of each.  At safety 0, use any function proclamation
information to type-and each value if the var is used as an argument
(i.e. reverse type propagation).  When encountering an if, push the
current value of each stack for each variable twice onto the same
stack, once for each branch of the if.  Determine if the test allows
either type to be type-anded with the corresponding implied type
(e.g. (if (consp x) ...) means to type-and 'cons into the type used
for the true branch, etc.)  At the exit of the if, pop the stacks
twice and set the type to the type-or of both branches.  When
encountering a setq, set the "current" type to the value of the form,
and the maximal type to the type-or of the form with the existing
maximal type.  When encountering a tagbody, and the first tag therein,
which should be the only case that the flow of control into the
following statement might not come from the statement above, push the
types again to the stack and evaluate the body.  If any variable's
type was changed, set it to T and repeat.  Iterate over this until no
variable's type is changed.  When finally done with the let body,
effectively autodeclare the maximal type.

This is just an outline, of course.  There may be a few other forms in
the tagbody camp, for example, and the logic needs more thought, but
something like this ought to work.  Paul?

Take care,


> It has committed a much larger amount of heap than last night  (315 Mbytes
> vs about 70) with much more room between the amount of memory actually in
> use and the committed virtual memory.  Usually the committed heap grows out
> to about 135 Mbytes quite quickly.
> 
> This implies that GCL is running more slowly because it is thrashing so I
> suppose the HEAD version needs to grab more heap at startup and/or else to
> grow more easily.
> 
> What is the best way to do this?
> 
> The HEAD GCL C stack start is bogus (it should be below the start of the
> program text).  The configure log gives CSTACK_ADDRESS = 0x22ffff.  I'll
> look further into that discrepancy later on.
> 
> Cheers
> 
> Mike Thomas
> ========================================================
> $ unixport/saved_ansi_gcl.exe
> GCL (GNU Common Lisp)  2.7.0 ANSI    Dec 30 2005 22:04:14
> Source License: LGPL(gcl,gmp,pargcl), GPL(unexec,bfd)
> Binary License:  GPL due to GPL'ed components: (UNEXEC)
> Modifications of this banner must retain notice of a compatible license
> Dedicated to the memory of W. Schelter
> 
> Use (help) to get some basic information on how to use GCL.
> 
> >(room)
> 
> WS      UP/MP     FI%      GC TYPES
> 
>  2     553/553  45.8%         CONS FIXNUM SHORT-FLOAT CHARACTER IFUN SPICE
>  8      87/120  65.6%         ARRAY PATHNAME BIT-VECTOR VECTOR HASH-TABLE
> STREAM
>  CCLOSURE CLOSURE
>  4      34/52   75.2%         STRUCTURE BIGNUM RATIONAL COMPLEX READTABLE
> CFUN
> 10     215/246  93.3%         SYMBOL
> 14       1/2    23.3%         PACKAGE
>  6      40/100  79.9%         SFUN LONG-FLOAT STRING RANDOM-STATE GFUN VFUN
> AFUN
>  CFDATA
> 
>        111/512                contiguous (81 blocks)
>            438                hole
>            194   0.0%         relocatable
> 
>        930 pages for cells
>       1673 total pages
>     251410 pages available
>       9061 pages in heap but not gc'd + pages needed for gc marking
>     262144 maximum pages
> 
> 
> Key:
> 
> WS: words per struct
> UP: allocated pages
> MP: maximum pages
> FI: fraction of cells in use on allocated pages
> GC: number of gc triggers allocating this type
> 
> word size:            32 bits
> page size:            4096 bytes
> heap start:           0x3000000
> heap max :            0x43000000
> shared library start: 0x77359400
> cstack start:         0x40000000
> cstack mark offset:   233 bytes
> cstack direction:     downward
> cstack alignment:     32 bytes
> cstack max:           2293759 bytes
> immfix start:         0x80000000
> immfix size:          1073741824 fixnums
> 
> >
> 
> 
> ==========================================================================
> 
> $ unixport/saved_ansi_gcl.exe
> GCL (GNU Common Lisp)  2.6.8 ANSI    Dec 23 2005 06:23:31
> Source License: LGPL(gcl,gmp), GPL(unexec,bfd)
> Binary License:  GPL due to GPL'ed components: (UNEXEC)
> Modifications of this banner must retain notice of a compatible license
> Dedicated to the memory of W. Schelter
> 
> Use (help) to get some basic information on how to use GCL.
> 
> >(room)
> 
>    949/949    36.5%         CONS RATIO COMPLEX STRUCTURE
>     21/28      9.9%         FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE
> READTABLE
> SPICE
>     93/404    83.0%         SYMBOL STREAM
>      1/2      20.5%         PACKAGE
>     73/373    61.4%         ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME
> CCLOSURE
>  CLOSURE
>     27/32     67.8%         STRING
>    325/451     1.6%         CFUN BIGNUM LONG-FLOAT
>     11/115    84.1%         SFUN GFUN VFUN AFUN CFDATA
> 
>    105/512                  contiguous (469 blocks)
>        13107                hole
>        5242    0.0%         relocatable
> 
>       1500 pages for cells
>      19954 total pages
>     101613 pages available
>       9505 pages in heap but not gc'd + pages needed for gc marking
>     131072 maximum pages
> 
> >
> 
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah




reply via email to

[Prev in Thread] Current Thread [Next in Thread]