Re: [Bug-gnubg] Guestimated abs. rating too high ?

[Joern Thyssen]

Hi Peter,

On Sun, Aug 04, 2002 at 04:17:35PM +1000, Peter Orum wrote

> It seems to me that perhaps the estimated ratings are too high, 
> perhaps by as much as 100 points. This is based on my own history 
> and many matches against GGRaccoon on GamesGrid.
> 
> For example, the match attached gives me a rating of 2038, and 
> GGRaccoon 2095. According to http://www.gamesgrid.com/faq-
> GGrobots.html, GGRaccoon's average rating is 1935, and the highest 
> it has currently been is 2072 according to its info page. It seems to me 
> its rating is generally between 1850 and 1950. GGRaccoon plays with 
> no lookahead. I would suggest that the Guestimated rating for the 
> attached match could be 1995 for GGRaccoon. 2095 is just too high.

Well, according to the attached file it plays with an error rate of
0.003 which is quite well.

> 
> Similarly for myself, on FIBS and GamesGrid my highest ratings have 
> been about 1930, and currently I am 1850 to 1880. I would suggest 
> that for this match, 1938 is more reasonable than 2038.
> 
> I am basing this on not this match alone. I would be happy to provide 
> more matches & statistics if you would like them. I think having the 
> Guestimated ratings based on GGRaccoon is a good way to relate 
> the error rate to actual achievable ratings, and ratings over 2000 
> should only apply to a perfect match.

The absolute rating is based on a, somewhat arbitrary, linear
interpolation scheme:

epm     rating
0.000   2200
0.005   2050
0.010   1950
0.015   1850
0.020   1750
0.025   1600
0.030   1500 
oo       500

We may very well shift the scale down to, say,

epm     rating
0.000   2050
0.005   1950
0.010   1900
0.015   1850
0.020   1750
0.025   1600
0.030   1500
oo       500

With this scale GGRaccoon would be rated at 1990 in the attached file.


What we really need is thousands of analysed matches to gather some
data 

(rating, error rate per move)

we can do some statistics on. One match is simply not enough. 

We could request users on rec.games.backgammon to send (rating, error
rate per move) from their matches on GamesGrid and FIBS. Once enough
data has been collected we can try to find a function that fits the
data.

Jørn

-- 
Joern Thyssen, PhD
Vendsysselgade 3, 3., DK-9000 Aalborg, Denmark
+45 9813 2791 (private) / +45 2077 2689 (mobile) / +45 9633 7036 (work)