[gnugo-devel] benchmarks

[Arend Bayer]

The files I used to benchmark the large scale patch are now in CVS,
in the new directory regression/benchmark.

I've attached the script that I used to generate them. I first gathered
12 .sgf-files from games against computer opponents (the 9 from Gifu
plus 3 more from Edmonton). From each, 20 consecutive of GNU Go's moves
will be replayed via reg_genmove (20 should be enough to get realistic
persistent cache effects). These 20 moves are taken in rotation from
1--40, 41--80, ... 201--240 such that each phase of the game is
equally represented.
(This is done automatically with my script as
sgf2gtp.py --auto 20 --auto-max 240 <files>.)
The only intervention by me is that I moved GnuGo-Martha into the
endgame phase, because otherwise 25% of the total run time would have
been spent in this game, against an opponent that we probably couldn't
care less about...

Fixing the move to 20 per .gtp file means that the absolute timings
retain some meaning.

Of course, one should extend this suite with games against human
opponents. Also, a script to get convenient output would be worthwile.

(I've just used the primitive shell script below for now.)

Arend


#!/bin/bash
if [[ -n $1 ]] ; then
        GNUGO=$1
else
        GNUGO=../../interface/gnugo
fi

# Make sure all files are in the disk cache
cat *.gtp >/dev/null

for file in *.gtp ; do
        echo "$file:"
        time $GNUGO --mode gtp --quiet <$file >/dev/null
done

sgf2gtp.py
Description: Text document

sgfparser.py
Description: Text document