Re: [Bug-gnubg] gnubg.bd

[Joseph Heled]

Joern Thyssen wrote:
> Hi all,
>
> I would very much like to do some changes to gnubg.bd. Of course, I'll
> try to keep backwards compability.
>
> Very shortly, gnubg will be able to read large one-sided databases
> and large two-sided databases. To keep the interface general I would
> like to split gnubg.bd into two separate files: one for the one-sided
> database and one for the two-sided database. For example, gnubg will use
> four databases:
>     - gnubg_osa.bd  (small one-sided bearoff database read into memory)
>     - gnubg_osb.bd  (huge one-sided bearoff database used from disk)
>     - gnubg_tsa.bd  (small two-sided bearoff database read into memory)
>     - gnubg_tsb.bd  (huge two-sided bearoff database used from disk)
>
> Also, gnubg_osa.bd will be re-generated to include gammon probabilities
> and gnubg_osb.bd will be re-generated to include cubeful equities for
> money game.
>
> An alternative to gnubg_osa.bd is to include it directly into the source
> as Josephs does with fibs2html/sagnubg. This will increase the
> executable size and memory footprint with up to 6 MB (can be decreased
> to 2 MB if my compression scheme is used). 

why increase? I don't distribute the source file just like gnubg don't 
distribute gnubg.bd. They are generated on the target machine. Both take space 
on the hard drive, one in binary form, one in source form, and both take space 
when the program is running. My approach was that the one sided database is not 
optional, i.e. you always want it, so there is no reason for an extra file.

And after stripping away the zero entries, the size is around 1.1M,

1> ls -l br1.o
-rw-r--r--   1 joseph   wheel     1126324 Aug 19 07:54 br1.o

> One can play the same trick with gnubg_osb.bd but the two-sided database
> is not as necesary as the one-sided which is used to calculate bg
> probabilities for the race net (and it will be used for one-sided
> rollouts).
>
> Once my code is finished we will get:
>
> - exact cubeful equities for money game (with the two-sided database)
> - huge one-sided databases (I'm not sure this is an improvement as the
>   race net is quite good)
> - gnubg for windows will be distributed with gnubg_osa.bd and
>   gnubg_tsa.bd, but programs will be included that can generate
>   arbitary-size bearoff databases for gnubg, only limited by the user's
>   patience and disk space.[*]

Does your plan includes gammon rates for bearoff positions, where one side has 
15 checkers?

-Joseph

> [*] Features of the new makebearoff programs:
>
> - uses a cache instead of malloc'ing the entire bearoff database
>    (this will of course slow down generation, but will make it possible
>    to generate huge databases without owning 1 GB of memory)
>   The cache can be improved a lot, for example, a better hash function
>   on position IDs is needed.
> - can use previosly generated databases. For example, the 12 pt
>   databases has 37M positions, but the 17M is already in the 11 pt
>   database. 
> - cubeful/cubeless two-sided bearoff databases with n chequers on m
>   points (1<=m<=6)
> - one-sided bearoff databases with bearoff and gammon distributions
>   (either exact distributions or distributions approximated by normal
>   distributions) for 15 chequers on m points (1<=m<=23), although the
>   12-14 points seems to be the practical limit. Also, the exact
>   distributions can be compressed in the file storing only non-zero
>   elements.
>
> So far I've generated the 12 pt one-sided database (approximated with
> normal dist). It's 300 MB and takes 15 hours to generate on my 1Ghz
> laptop. 
>
> With respect to one-sided bearoff databases, it might be possible to
> generate some hybrids, e.g., less than 5 chequers on the outfield is 18M
> positions. I'll investigate that.
>
> Jørn