Re: [gnugo-devel] 9x9 opening stats against humans

[Douglas Ridgway]

On Fri, 9 Jul 2004, Gunnar Farnebäck wrote:

> How big set of pro games do you have? (Just curious, it does of course
> not matter for this conclusion.)

Around 300, with something like half from Gobase. It'd be nice to have
more. I know GoGoD has a hundred, but I don't have a copy, and I don't 
know the degree of overlap.

> [Doug wrote:]
>
> >   * The move weighting, based on popularity in the original training set,
> > seems problematic. First, since a deweighted move is played more rarely,
> > it takes more games to figure out if it's better or worse than the higher
> > rated moves.
> 
> That's problematic as far as evaluation goes, but I don't think it's a
> direct playing strength problem.

Agreed.

> > Second, the more moves are weighted, the more predictable they
> > become.
> 
> But the higher weighted moves let us stay in the fuseki database
> longer, usually reducing the predictability.

Okay, I see, because GnuGo is (in general) so predictable on its own.

> > I hope to figure out how to add the pro game patterns to the existing set.
> 
> It's probably easier and/or better to recompile the database from
> scratch, assuming you have or can get a large enough training set. You
> might want to patch extract_fuseki.c to give higher weight to the pro
> games.
> 
> > Would anyone mind if I changed the weights at the same time?
> 
> Probably not, although it might depend on by what principles you
> intend to modify them. :-)

That's the hard part. Even straightforward stuff like "Discard
statistically worse moves" and "Copy the pros" can be argued with. Suppose
a move loses because it exposes a bug? You'll play stronger if you don't
play it, but once you stop playing it, you'll never know if you fix the
bug. And copying pros could make you worse, not better, if you can't 
follow up.

I suppose a conservative choice might be to try to maximize
unpredictability (not as easy as equal weighting, as you pointed out) and
to examine statistically bad patterns by hand to see what's wrong with
them before zeroing them out. Alternatively, some kind of weighting based 
on results that haven't reached significance.

> The current fuseki databases were created by MÃ¥ns Ullerstam three
> years ago. He is no longer on the list but you can try to mail him
> privately to see if he still has the games around.

Unfortunately, he didn't keep them. They were from IGS, and I can't find
them on their FTP site, so perhaps they are no longer available.

doug.
address@hidden