[Bug-gnubg] GNU Backgammon on backgames

[Øystein O Johansen]

http://www.bgsnowie.com/backgammon/articles.dhtml?id=56

I can see at Snowie's web site that Johannes Leverman talks nice about
Snowie's 4 backgame improvements to Snowie 3. I could not resist. I entered
these positions into GNU Backgammon, just to see how our little neural net
rates these positions.

The first position Johannes shows is this one:

    GNU Backgammon  Position ID: YBYScDfYtg0HAA
                    Match ID   : cIkUAAAAAAAA
    +24-23-22-21-20-19------18-17-16-15-14-13-+  O: player1
    |                O |   |    O  O        X |  0 points
    |                O |   |    O           X |
    |                  |   |                X |
    |                  |   |                  |
    |                  |   |                  |
    |                  |BAR|                  |v (Cube: 1)
    |                  |   |                  |
    |                  |   |                  |
    |    O  O          |   |                  |
    | O  O  O  X  X  X |   | X  X  X          |  Rolled 15
    | O  O  O  X  X  X |   | X  X  X  O     O |  0 points
    +-1--2--3--4--5--6-------7--8--9-10-11-12-+  X: player2


O has a very strong backgame and X is not strong enough to double. Snowie 3
gets this wrong. Snowie 4 gets this right. GNU Backgammon also gets this
right.

GNU Backgammon:
          Win  W(g) W(bg) L(g)  L(bg)  Equity  (cubeful)
static: 0.579 0.423 0.109 0.008 0.000 (+0.682  (+1.000))
 1 ply: 0.575 0.434 0.120 0.006 0.000 (+0.696  (+1.000))
 2 ply: 0.524 0.381 0.122 0.008 0.000 (+0.543  (+0.779))

No double           : +0.779
Double, pass        : +1.000   (+0.221)
Double, take        : +0.762   (-0.017)

Correct cube action: No double, take

Of course I also showed this position to JellyFish. JellyFish says
double/drop and a cubeless equity of +0.840 which agrees a lot with Snowie
3.

The second position Johannes shows in this article, is this:

    GNU Backgammon  Position ID: cLeDASTgc3CAGQ
                    Match ID   : cAkFAAAAAAAA
    +24-23-22-21-20-19------18-17-16-15-14-13-+  O: player1
    |    X     X  O  O |   | O  O           X |  0 points
    |    X     X  O  O |   | O  O           X |
    |             O  O |   |    O           X |
    |                  |   |                  |
    |                  |   |                  |
    |                  |BAR|                  |v (Cube: 1)
    |                X |   |                  |
    |                X |   |                  |
    |                X |   |    X             |
    |                X |   |    X           O |  Rolled 21
    | O     O        X |   |    X           O |  0 points
    +-1--2--3--4--5--6-------7--8--9-10-11-12-+  X: player2

He is here trying to demonstrate how Snowie 4 makes backgame strategies, by
keeping the backgame plan.

Johannes and Snowie 4 agrees that 6/5 6/4 is the best move, while Snowie 3
says 23/22 23/21 since this move avoids the backgame. GNU Backgammon also
get this right at 2-ply and plays 6/5 6/4. The funny thing is JellyFish. At
level 5 (0-ply) it says: 6/3*. It does not even consider 23/22 23/21 as a
candidate. At level 5 6/5 6/4 is on fifth place. Reevaluating at level 7,
it still says 6/3*, but now 6/5 6/4 ranks as number 2, with a cl equity
difference of only 0.02.

GNU Backgammon:

    1. Cubeful 2-ply    6/5 6/4                      Eq.:  -0.815
       0.369 0.059 0.000 - 0.631 0.306 0.024
    2. Cubeful 2-ply    13/11 6/5                    Eq.:  -0.848 ( -0.033)
       0.350 0.066 0.001 - 0.650 0.290 0.021
    3. Cubeful 2-ply    6/3*                         Eq.:  -0.858 ( -0.043)
       0.341 0.064 0.001 - 0.659 0.274 0.018
    4. Cubeful 2-ply    23/22 23/21                  Eq.:  -0.883 ( -0.068)
       0.309 0.059 0.001 - 0.691 0.217 0.008

The last position, Johannes tries to demonstrate anti-backgame strategies.

    GNU Backgammon  Position ID: qHNwABvY3QYDIA
                    Match ID   : cAkaAAAAAAAA
    +24-23-22-21-20-19------18-17-16-15-14-13-+  O: player1
    | X        O  O  O |   |    O           X |  0 points
    |                O |   |    O           X |
    |                O |   |    O             |
    |                  |   |                  |
    |                  |   |                  |
    |                  |BAR|                  |v (Cube: 1)
    |                  |   |                  |
    |                  |   |                  |
    |             X  X |   |                O |
    |    O  O  X  X  X |   | X  X           O |  Rolled 46
    |    O  O  X  X  X |   | X  X           O |  0 points
    +-1--2--3--4--5--6-------7--8--9-10-11-12-+  X: player2

He writes that hitting is a blunder, and Snowie 3 hits but Snowie 4 does
not hit. Unfortunately GNU Backgammon hits as well does.

    1. Cubeful 2-ply    24/20* 13/7                  Eq.:  +0.920
       0.616 0.368 0.043 - 0.384 0.035 0.001
    2. Cubeful 2-ply    24/20*/14                    Eq.:  +0.912 ( -0.008)
       0.622 0.350 0.038 - 0.378 0.032 0.001
    3. Cubeful 2-ply    24/14                        Eq.:  +0.866 ( -0.054)
       0.661 0.254 0.017 - 0.339 0.033 0.001

The rollout really shows this is a blunder.

    1. Rollout          24/14                        Eq.:  +0.809
       0.632 0.229 0.011 - 0.368 0.053 0.001 CL  +0.450 CF  +0.809
      [0.002 0.002 0.000 - 0.002 0.001 0.000 CL   0.006 CF  +0.019]
        Truncated cubeful rollout (depth 15) with var.redn.
        648 games, seed 1148792576 using Mersenne Twister
        Play and cube: 0-ply cubeful [expert]
    2. Rollout          24/20*/14                    Eq.:  +0.691 ( -0.117)
       0.553 0.299 0.022 - 0.447 0.056 0.001 CL  +0.370 CF  +0.691
      [0.002 0.002 0.001 - 0.002 0.001 0.000 CL   0.007 CF  +0.020]
    3. Rollout          24/20* 13/7                  Eq.:  +0.689 ( -0.119)
       0.547 0.309 0.027 - 0.453 0.058 0.001 CL  +0.371 CF  +0.689
      [0.002 0.002 0.001 - 0.002 0.001 0.000 CL   0.007 CF  +0.023]

JellyFish play 24/14 (without the hit) as well even at 0-ply. But the
hitting play 24/20*/14 is only 0.004 cl equity points behind. Shame on GNU
Backgammon in this position.

Based on these three positions, which of course is not enough material to
make a general statement, I would rank the backgame abilities of the bots
in this order:

1. Snowie 4
2. GNU Backgammon
3. JellyFish 3.01
4. Snowie 3

-Øystein


-------------------------------------------------------------------
The information contained in this message may be CONFIDENTIAL and is
intended for the addressee only. Any unauthorised use, dissemination of the
information or copying of this message is prohibited. If you are not the
addressee, please notify the sender immediately by return e-mail and delete
this message.
Thank you.