Re: [gnubg] Help with a new MET

[Timothy Y. Chow]

On Wed, 13 Nov 2019, Joseph Heled wrote:
> I thought it was clear that what we want to establish a difference 
> between two (say players X and Y) by running games and testing that 
> gammon-rate(X) != gammon-rate(Y).

Thanks for the clarification.

If you're pitting X against Y and your null hypothesis is that their 
playing abilities are identical, then what I'd recommend is something like 
this.  Collect two sets of samples.  In the first sample, play a bunch of 
games and form a vector of counts like this:

(X wins BG, X wins G, X wins S, Y wins S, Y wins G, Y wins BG)

In the second sample, play the same number of games, and form a similar 
vector but reverse the order of the counts:

(Y wins BG, Y wins G, Y wins S, X wins S, X wins G, X wins BG)

Now you've reduced your problem to testing whether these two sets of 
sample vectors come from the same distribution.  There are various ways 
one can do this, a standard one being a chi squared two-sample test.

https://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/chi2samp.htm

Tim