Re: [Gcl-devel] 2.6.2 benchmarks

[Camm Maguire]

Greetings, and thanks as always, Mike!

"Mike Thomas" <address@hidden> writes:

> Hi Camm.
> 
> I ran the Gabriel benchmarks patched per your original email (ie, not
> including changes which might have been applied after Paul's suggestions)
> and with the makefile modified to use Windows CLISP 2.30's "-x" argument
> rather than piping to a "-" command-line (which did not work).
> 
> For Corman CL 2.5 I had to do each benchmark by hand as I couldn't get the
> ":quit" command to work with stdio from the makefile - in other words, the
> Corman results deserve to be taken with a substantial grain of salt as all
> output goes directly to the console rather than to a file.
> 
> Needless to say there is no CMUCL/SBCL on Windows.
> 
> I didn't see your test machine specifications, but my results (compared with

Dual Intel Xeon 2.4Ghz, 512Mb ram.

> CLISP) are broadly concordant with your's on Linux except for GCL
> input/output which is markedly worse on Windows - this could skew the times
> for tests with 1600 repeats or with small computation:I/O ratios.
> 

Agreed.  Please see my separate post on this.  Shortly at 

http:/people.debian.org/~camm/gabriel.tgz

I'll post the slightly modified suite I'm using.  The i/o issue should
be somewhat mitigated as I've removed the (print (time ...)) around
each test.  Beyond this, of course, it sure would be nice to rewrite
these tests to do fewer iterations on bigger problems, but I don't
have time for this.  

The original results have been run without sgc.  I will see what
effect this has.  Of course if there is one, this will be a
performance difference with windows too.

> The GCL timer granularity on my machine is roughly +/- 0.02 seconds, so
> accumulating runtime benchmark results of that magnitude with Windows GCL
> may be inaccurate eg:
> 

The test is not designed to accumulate these numbers, but rather
brackets the entire iteration with get-internal-run-time calls. 

> >(time (sleep 0.025))
> 
> real time       :      0.033 secs
> run-gbc time    :      0.033 secs
> child run time  :      0.000 secs
> gbc time        :      0.000 secs
> NIL
> 
> >(time (sleep 0.025))
> 
> real time       :      0.017 secs
> run-gbc time    :      0.017 secs
> child run time  :      0.000 secs
> gbc time        :      0.000 secs
> NIL
> 
> >(time (sleep 0.3))
> 
> real time       :      0.317 secs
> run-gbc time    :      0.317 secs
> child run time  :      0.000 secs
> gbc time        :      0.000 secs
> NIL
> 
> >(time (sleep 0.3))
> 
> real time       :      0.300 secs
> run-gbc time    :      0.300 secs
> child run time  :      0.000 secs
> gbc time        :      0.000 secs
> NIL
> 
> Cheers
> 
> Mike Thomas.
> 
> 
> Using a 2.4 GHz 512 Mb PI with Windows XP:
> 
> ============================================================================
> Test     GCL 2.62 (s) CLISP 2.30 (s) Corman 2.5 (s) (CLISP/GCL) (Corman/GCL)
> ===================================================+========================
> BOYER       7.450       107.703         15.861     |   14.5         2.1
> BROWSE     15.000        NA            134.724     |   NA           9.0
> CTAK        4.250        12.047          9.124     |    2.8         2.1
> DDERIV      5.700        24.625          7.192     |    4.3         1.3
> DERIV       4.867        22.953          6.731     |    4.7         1.4
> DESTRU-MOD  5.683        72.656         19.394     |   12.8         3.4
> DESTRU      8.617        73.047         18.954     |    8.5         2.2
> DIV2        5.033        31.328          6.670     |    6.2         1.3
> FFT-MOD     0.650       139.047         NA         |  213.9        NA
> FFT         0.983       138.781        196.118     |  141.2       199.5
> FPRINT      17.267       29.281        204.095     |    1.7        11.8
> FREAD       9.833        10.750        143.479     |    1.1        14.6
> FRPOLY      20.900      162.437         72.190     |    7.8         3.5
> PUZZLE-MOD  3.267       152.344         61.768     |   46.6        18.9
> PUZZLE      3.583       152.125         62.532     |   42.5        17.5
> STAK        6.817        68.203         NA         |   10.0        NA
> TAK-MOD     2.667        85.625          4.654     |   32.1         1.7
> TAK         3.083        83.656          4.527     |   27.1         1.5
> TAKL        1.000        55.047          2.745     |   55.0         2.7
> TAKR        2.250        38.703          6.777     |   17.2         3.0
> TPRINT     13.033        32.578        528.728     |    2.5        40.6
> TRAVERSE   36.300       880.406        199.521     |   24.3         5.5
> TRIANG-MOD 48.900      2067.016        862.122     |   42.3        17.6
> TRIANG     52.283      1692.109        476.243     |   32.4         9.1
> ===================================================+========================

These are looking pretty good.  They might be worth posting in the
release notes too after they've been given a thorough skeptical
working over, preferably by people who know clisp/windows and
corman/windows. 

Anyone know why clisp is not compiling the browse test (and corman the
others?) 

Take care,

> 
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah