Re: [Bug-gnubg] Feature Request: Gnu to use rollouts for playing

[Massimiliano Maini]

Ideal would be just to implement the feature and make it configurable.
Gnubg should be able to play (and do rollouts ?) at a configurable
rollout level.
Establishing what level (where to truncate, ply during play, how to
stop , VR on/off)
is good enough may be a personal matter. If the feature is available and
configurable, we should just agree on its default setting (which may
as well be the
same as XG, for comparison purposes).

MaX.

2009/9/18 Michael Depreli <address@hidden>:
> I think if you're going to introduce this feature then what would impress
> the masses is a more accurate version
> of XGRoller rather than something based on speed (within reason).
>
> My tests using an @6 rollout WITH VR combined with stop on jsd produced some
> very impressive results
> for checker play comparison rollouts. (Relative equity rather than absolute)
>
> I haven't checked out cube decisions but it did occur to me that it wasn't
> necessary to have the
> same settings for cube and checker rollouts if it could be shown that
> different settings were better for each.
>
>
>
>> Subject: RE: [Bug-gnubg] Feature Request: Gnu to use rollouts for playing
>> Date: Fri, 18 Sep 2009 18:21:02 +0100
>> From: address@hidden
>> To: address@hidden; address@hidden
>>
>>
>> Michael Depreli wrote:
>>
>>
>> Are you sure that for 0-ply switching off VR is more accurate in
>> a given time?
>> Unless I'm doing something wrong I get the opposite results.
>>
>>
>> Michael, its me that's doing it wrong. I mis-read the SE output. Here is
>> a revised test (still only short), showing that VR leads to slightly
>> LOWER SE for the same rollout period.
>>
>> Time Without VR With VR
>> (sec) Trials SE Trials SE
>> ===============================================
>> 60 8817 0.033 376 0.024
>> 120 16621 0.023 744 0.021
>> 180 25356 0.019 1112 0.017
>> 216 31131 0.017 1296 0.016
>>
>> So, at 0-ply, using VR or not is a wash. VR might be slightly better in
>> terms of a low SE, but to impress the masses it might be optimal to
>> choose no VR and lots of trials in a short time.
>>
>> -- Ian
>>
>> > Subject: RE: [Bug-gnubg] Feature Request: Gnu to use rollouts
>> for playing
>> > Date: Fri, 18 Sep 2009 13:12:12 +0100
>> > From: address@hidden
>> > To: address@hidden
>> >
>> >
>> >
>> > I've finally got around to running some tests.
>> >
>> > Test Environment:
>> > Intel Core2 Duo T8100 @ 2.10 GHz, 2 GB RAM Windows XP
>> Professional GNU
>> > Backgammon 0.90-mingw 20090914 Cache 5 MB
>> >
>> >
>> > The test position was opening 43: 13/9 13/10 in a money game
>> > 1296 trials, cubeful evaluation, truncation at database
>> (default
>> > databases installed)
>> >
>> > Thr VR Plies Trials/min Time
>> > ======================================
>> > 2 No 0 7776 00:00:09
>> > 2 Yes 0 390 00:03:30
>> > 1 No 0 4860 00:00:16
>> > 1 Yes 0 205 00:06:22
>> > 2 No 1 200 00:06:20
>> > 2 Yes 1 131 00:08:59
>> > 1 No 1 103 00:12:04
>> > 1 Yes 1 74 00:18:18
>> > 2 No XGR 15552 00:00:05
>> > 2 Yes XGR 1897 00:00:41
>> >
>> > Thr = number of threads
>> > XGR = Extreme Gammon XGRoller emulation: 0 ply rollouts
>> truncated at 6
>> > moves, first two cube decisions at 1-ply.
>> > Extreme Gammon does not use VR for XGRoller, but I have
>> repeated the
>> > test with VR on.
>> >
>> > It is clear that the speed hit is enormous when using VR for
>> 0-ply
>> > rollouts, by a factor of around 20.
>> >
>> > At 1-ply, the speed hit is still noticeable, but only by a
>> factor of
>> > around 1.5.
>> >
>> > At 5 seconds per move, the XGR setting is certainly viable for
>> play and
>> > analysis. And reading the BGOnline forums, there certainly
>> seems to be a
>> > demand for this feature. (There is no reason for gnubg to copy
>> XG's
>> > exact settings, if we think something else is superior. In
>> fact, I think
>> > it should be possible to specify a .rol file to use so that
>> people can
>> > choose. The default option, though, should be FAST.)
>> >
>> > High speed is great, but the reliability of the result is also
>> important
>> > (though oft overlooked). I also tested the Standard Error
>> reported after
>> > one minute for various settings. All tests used two threads.
>> >
>> > VR Plies Trials SE
>> > =============================
>> > Yes 1 136 0.016
>> > No 1 206 0.088
>> > Yes 0 373 0.016
>> > No 0 8836 0.002
>> >
>> > It appears that to achieve a low SE in a given time, VR is
>> worthwhile
>> > for 1-ply rollouts, but for 0-ply rollouts it is better to
>> specify a
>> > much larger number of trials and switch off VR.
>> >
>> >
>> > -- Ian
>> >
>> > > -----Original Message-----
>> > > From: Christian Anthon [mailto:address@hidden
>> > > Sent: 21 August 2009 22:33
>> > > To: Ian Shaw
>> > > Cc: address@hidden
>> > > Subject: Re: [Bug-gnubg] Feature Request: Gnu to use
>> rollouts for
>> > > playing
>> > >
>> > > Can you check the speed of the threaded/non-threaded
>> rollouts (say
>> > > 0ply eval 6ply truncation).
>> > >
>> > > Christian.
>> > >
>> > > On Fri, Aug 21, 2009 at 6:54 PM, Ian
>> > > Shaw<address@hidden> wrote:
>> > > >
>> > > >
>> > > >> -----Original Message-----
>> > > >> From: Christian Anthon
>> [mailto:address@hidden
>> > > >> Sent: 20 August 2009 07:28
>> > > >> To: Ian Shaw
>> > > >> Cc: address@hidden
>> > > >> Subject: Re: [Bug-gnubg] Feature Request: Gnu to use
>> rollouts for
>> > > >> playing
>> > > >>
>> > > >> Most of the code is already in place. Things to consider:
>> > > >>
>> > > >> a) move filter - i.e. which positions to roll out
>> > > >
>> > > > I would expect to be able to specify these, possibly by
>> > > loading a .rol
>> > > > file.
>> > > >
>> > > >> b) using GNURoller for plays and cubes in rollouts
>> > > >
>> > > > I have to think about this some more. In theory, why not?
>> > > However, I
>> > > > do wonder whether it is equivalent to a normal rollout of
>> > > certain settings.
>> > > > As I said, I haven't considered this.
>> > > >
>> > > >> c) efficiency of the current mt code for fast games in
>> > > roll outs - we
>> > > >> lock all threads during accounting
>> > > >
>> > > > I'm not up with the mt. Isn't it only active in rollouts?
>> > > It might be
>> > > > worth investigating whether you can multithread at the
>> > > level of the nn
>> > > > evaluation of a single position, which with all those SSE
>> > > calls surely
>> > > > takes up the most time. This would speed up play, analyses
>> and
>> > > > rollouts to the same degree. You'd have to get an opinion
>> > > from the mt
>> > > > guru's as to whether this is feasible and wouldn't incur
>> > > too much overhead.
>> > > >
>> > > >> d) variance reduction - my limited experience tells me
>> that
>> > > >> variance/time is more or less independent of turning it
>> > > on/off for 0
>> > > >> ply rollouts
>> > > >>
>> > > > I've just run a simple test on a single position using a 0
>> > > ply rollout.
>> > > > Without VR I recorded 589 moves in 1 minute; with VR it
>> was
>> > > 34 moves.
>> > > >
>> > > > It stands to reason that VR would slow things down. With
>> no
>> > > VR gnubg
>> > > > only needs to evaluate the actual roll. For VR, gnubg must
>> evaluate
>> > > > the outcome all possible rolls to see how lucky the actual
>> roll was
>> > > > compared to the others. This must be roughly equivalent to
>> a 1-ply
>> > > > lookahead in cost.
>> > > >
>> > > > On a 1-ply rollout, the difference is less pronounced: 21
>> > > trials in 1
>> > > > minute with VR and 30 without VR. I suppose this is the
>> effect of
>> > > > caching.
>> > > >
>> > > > -- Ian
>>
>>
>
> ________________________________
> Upgrade to Internet Explorer 8 Optimised for MSN. Download Now
> _______________________________________________
> Bug-gnubg mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/bug-gnubg
>
>