Re: [Bug-gnubg]

[Douglas Zare]

Quoting Joseph Heled <address@hidden>:

> > Isn't that less than 1.5 standard deviations? Or do you mean cubeless money
> > games?

> Yes. those are cubeless money games.

In that case, the result is more statistically significant, and means a larger 
difference in performance. 

> I thought the STD is 
> sqrt(1/(p*(1-p)*N), and assuming p ~= 0.5, this gives 0.0007
> 
> Perhaps I am wrong. 

That is roughly the standard deviation of the number of wins. However, the 
standard deviation in the points per game is larger by more than a factor of 2, 
since a single loss is -1 rather than 0, and there are gammons and backgammons.

> How many games would you deem necessary?

That depends on the purpose. If you want to conclude that 14 plays better than 
13 on 0-ply with no cube, you probably have enough evidence to be very 
confident of that. If you want to figure out how much stronger the 0-ply 
evaluations are, the range of nonsurprising possibilities is still pretty large 
compared with the advantage.

Douglas Zare