Re: [Bug-gnubg] Results from the bot shootout

[Joseph Heled]

With gnubg/Jelly we have p = .593 (estimated win percentage) and N =
1000. The 95%  confidence interval is [.562,.623], so we can be sure
gnubg is better.

gnubg/bgb p=.519 and the interval is [.487,.55] so you can't say gnubg
is better than any of the others.

with p around 52% percent and 10,000 matches you get [0.509, 0.528],
which enables you to claim gnubg is better with 95% probability.

-Joseph

On 7/5/07, Achim Mueller <address@hidden> wrote:
> * Øystein Johansen <address@hidden> [070705 09:23]:
> > Achim Mueller wrote:
> > >>From GammonU:
> > >
> > >              BGB    Jellyfish  Snowie 4.5  GnuBG 0.14
> > > BGB           -        534        504          481
> > > Jellyfish    466        -         428          407
> > > Snowie 4.5   496       572         -           489
> > > GnuBG 0.14   519       593        511           -
> > >
> >
> > It's 25 point matches, isn't it?
> >
> > It looks like GNU Backgammon beats them all. Is that the right
> > interpretation?
> >
> > This was about the result I would expect in a match series. The point is
> > that the positions where GNU Backgammon outperforms Snowie is more
> > frequent than the positions where Snowie outperforms GNU Backgammon.
> >
> > Would anyone estimate the standard deviation of these numbers?
>
> If sqrt(a*b/1000) is correct than it's:
>
>
>                 std.err   sigma      confidence
> gnubg/bgb    =   15.79     2.28  ==>  98,8%
> gnubg/snowie =   15.81     1.39  ==>  91,8%
> gnubg/jelly  =   15.53    11.98  ==> 100,0% (well, nearly)
>
> snowie/bgb   =   15.81     0.51  ==>  69,5%
> snowie/jelly =   15.64     9.21  ==> 100,0%
>
> bgb/jelly    =   15.77     4.31  ==> 100,0%
>
>
> So we can be quite sure that jellyfish will be beat by all other
> bots, we can be quite confident that gnubg is better than bgb, we can be
> a little confident that gnubg is better than snowie.
>
> Am I right ith these numbers?
>
> Ciao
>
> Achim
>
>
>
> _______________________________________________
> Bug-gnubg mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/bug-gnubg