[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug-gnubg] TEST RESULT: Variance Reduction
From: |
Ian Shaw |
Subject: |
[Bug-gnubg] TEST RESULT: Variance Reduction |
Date: |
Mon, 30 Jun 2003 11:15:20 +0100 |
I've re-tested rollouts of the initial position now that the variance reduction
has been corrected. I'd like to get some comment on before posting the test to
GammOnLine.
My observations are:
1) Wins reduced
The winner of the opening roll wins less often. This is to be expected since
doubles would always be a good opening roll. The old VR was factoring in some
rolls being doubles and overcorrecting for what it saw as a poor set of rolls
during the rollout.
2) Crawford 2-away Match Equity
Leader's MEQ at Crawford 2-away is even lower than the original rollout
suggests, 68.2% rather than 68.3%. At 2-ply, my rollouts suggested that Trailer
benefited even more, so we can expect Leaders' MEQ to drop below my current
best value of 67.7%, which is already lower than any published MET. (Oh how I
want to get multi-processing before I repeat THAT rollout!)
3) Standard Error
The reported overall std error is a LOT lower (see result below). I would have
expected some reduction because, without doubles, there is simply less
variation in the opening rolls. There is a huge swing on 66 for starters - it
wins about 2/3 of games (according to 2-ply evaluation).
Even so, I was surprised to find such a large reduction in the std error. The
individual std errors for w, wg etc have not changed much, so it's hard to see
how the overall value would change so much. Could you have a look at this,
please.
4) Rotating opening rolls
I rolled out 7776 times (6 * 36 * 36). Then I wondered how the stratification
is affected when it's the opening roll. Will the rollout have done any
stratification at all? Should I have done a multiple of 21 * 36 = 756? Will the
stratification work if I do?
5) Match Equity Calculation
Finally, I'm a bit suspicious of the reported match equity values. I calculated
them by hand from the win percentages, and got different results.
Match equities reported by gnubg and calculated by hand:
Wrong variance reduction
Trailer: 33.784% 33.635%
Leader: 69.939% 70.265%
Corrected variance reduction
Trailer: 69.081% 69.080%
Leader: 69.081% 69.080%
The ones from the corrected variance reduction rollout look OK, with the
discrepancy attributable to rounding error. The earlier results look suspect
though. I don't think rounding error would give such a large discrepancy. (E.g.
the 70.265 value calculates to 70.25 and 70.5 if I round the win percentages to
3 and 2 dps, so I can't see how it can get from 69.939 to 70.265.)
Leader's MEQ calculated as w + (l - lg) * 0.5
Trailer's MEQ calculated as wg + (w - wg) * 0.5
Rollout results
0-ply trials/side 7776 Score: Crawford 2-away
Wrong variance reduction
1st Roll w wg wbg l
lg lbg cubeless cubeful
Trailer 0.5258 0.1469 0.0186 0.4742 0.1923 0.0300 33.784% 33.784%
std err 0.0010 0.0009 0.0010 0.0023 0.0013 0.280% 0.280%
Leader 0.5280 0.2233 0.0324 0.4720 0.1227 0.0124 69.939% 69.939%
std err 0.0010 0.0024 0.0013 0.0009 0.0008 0.280% 0.280%
Leader's MEQ
Combined 0.5011 0.2078 0.0312 0.4989 0.1348 0.0155 68.315%
Combined Std Err 0.198% 95%CI
0.388%
Corrected variance reduction
1st Roll w wg wbg l
lg lbg cubeless cubeful
Trailer 0.5114 0.1424 0.0183 0.4886 0.2001 0.0299 32.687% 32.687%
std err 0.0010 0.0008 0.0010 0.0023 0.0013 0.069% 0.069%
Leader 0.5123 0.2176 0.0323 0.4877 0.1307 0.0138 69.081% 69.081%
std err 0.0010 0.0023 0.0013 0.0008 0.0009 0.069% 0.069%
Leader's MEQ
Combined 0.5005 0.2089 0.0311 0.4996 0.1366 0.0161 68.195%
Combined Std Err 0.049% 95%CI
0.096%
--Ian
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Bug-gnubg] TEST RESULT: Variance Reduction,
Ian Shaw <=