Re: [Bug-gnubg] Snowie error rates versus gnubg error rates

[Joern Thyssen]

On Sun, Apr 06, 2003 at 11:33:18PM -0400, Douglas Zare wrote
> Quoting Joern Thyssen <address@hidden>:
> 
> > On Sun, Apr 06, 2003 at 05:36:34PM -0300, Albert Silver wrote
> > > I mention this because it just
> > > seems to me, from empirical experience alone, that 0.0083 as the bottom
> > > limit of Expert and 0.012 as the bottom limit of Advanced seems a little
> > > strict. I know Snowie has in practice been stricter in its grading, but
> > > I didn't get the impression it was THAT strict. 
> > 
> > Apperently it is :-) With the new threshold gnubg should be, on average,
> > equally strict as Snowie.
> 
> I think Snowie's thresholds are not strict enough. In my opinion,
> there are too many players with average error rates under 4.4 mppm for
> them all to be considered world-class. I favor decreasing the limit to
> perhaps 3.5 mppm by Snowie's measure. 
> 
> Otherwise, some people rated as "world-class" by bots will be rated
> "pigeon" by substantially stronger human players. I just analyzed a
> money session in which a player with an error rate under 3.0 (by
> Snowie rollouts) was estimated to be the favorite by about 0.12 ppg
> against a player with an error rate of 4.4. 

0.12ppg seems like a lot when there is only 1.4 millipoints in
difference in the error rates. Were the games very long? On a 1-cube you
would need an average of 80 decisions per player to reach 0.12 ppg, so I
assume one of the players made a lot of errors on high cubes?

> I don't feel strongly about the levels at which someone is termed advanced 
> versus expert or walking bye. 
> 
> Zbot will give people the option of using a different normalization
> than EMG.  That opens another can of worms, but it is my way of trying
> to make it not much easier to play at a WC level in short matches than
> for money play. You may wish to test this under your definition.

Are you able to discuss these other normalisations -- or are they
considered "trace secrets" so far :-)

Joern