Re: [gnugo-devel] Go as SPEC benchmark?

[Arend Bayer]

Dear James Koehler,

personally I, too, would find it very nice to have a Go program or
specifically GNU Go in the spec benchmark suite. Without knowing much
about professional benchmark requirements, I'd also think that it is
well suited as a benchmark, as the performance critical code is spread
among many very different functions.
Also, we do aim at complete reproducability among different platforms,
so additional testing with respect to this in the course of getting
it accepted for SPEC would be very welcome.

I see some specific problems, however, maybe you can comment on these:

1. About 5% of the time is spent doing floating point computations
(whereas the rest of the code is 98% integer computations). Is it a
pre-condition for SPEC acceptance to depend either completely on integer
or completely on floating point performance?

(It is not much code and could be re-written using fixed-point integer
arithmetic.)

2. The cache is badly organized for 64bit architectures (i.e. they can
store less entries if using the same amount of memory). I don't know
what kind of fairness between 32 and 64 bit architectures is expected
for SPEC.

Definitely, one would have to make sure that this problem doesn't affect
behaviour (which it could at the moment), as this would add serious
noise to the benchmark results.


I am also curious what others in our team think. I'd see sufficient
overlap with our general goals to make this worth some reasonable
effort.

Regards,
Arend


/---------------------------------------------------------------------\
|                             Arend Bayer                             |
|                          Flodelingsweg 27a                          |
|                             53121 Bonn                              |
|                            0228-9813803                             |
\---------------------------------------------------------------------/