[Bug-gnubg] Re: GnuBG: Fractional-ply evaluators

[Tom Keith]

Hi Nis.

Thank you for your thoughful reply.

Nis Jorgensen wrote:
> A little while ago, I suggested on the bug-gnubg list an explanation of
> why 1-ply is off the mark. 

I'm not sure it is accurate to say that 1-ply is more off the mark 
than 0-ply.  According to the test I did, 1-ply is better at
estimating absolute equity than 0-ply.  And I've heard that 1-ply
plays better than 0-ply too.  At least in general.  

The issue seems to be that there are certain classes of positions 
where 0-ply is significantly better than 1-ply.

> Here is the important part:
> 
[clip]
> 
> Note that this was written in the context of someone claiming that 0-ply
> is overrating positions. I do not think this is the case. The general
> principle holds: On average, 1-ply rates positions lower than 0-ply

On the positions in my sample, 0-ply underestimates equity on average,
and 1-ply overestimates.
            
             Average Equity
  0-ply:      -0.017
  1-ply:      +0.045
  rollout:    +0.012

> It would be nice to know the size of your sample.

150,000 positions

> I think I aske  Joseph to make his
> benchmark available some time ago, and I would like to repeat the
> request. If possible in some semi-readable format ...
> 
> The same goes for your sample of positions.

I'd be happy to make my rollouts/evaluations available.

> The pipcount is strongly related to the gwc. Could you perhaps check
> if the correlation  between the 0-ply eval and  BestPly is stronger or
> weaker than between pipcount and BestPly? (BestPly is 0 if 0-ply is
> best, 1 if 1-ply is).

  Correlation between pipcount and BestPly = -0.190257
  Correlation between 0-ply GWC and BestPly = 0.618061

> Bonus question: Find a best fit between "true equity" and 
> p1 + (a * p0 + b)(p0- p1)

What is this expression?
 
> Just to clarify: What does the hybrid do when the pip-count is between
> -40 and 40? Is it then using 0.5-ply?
> What happens above 150 and below -160?

Outside the given ranges, use 0.5-ply:

  if ( -160<rpc && rpc<-40 ) return eval1 ;
  if (   40<rpc && rpc<150 ) return eval0 ;
  return 0.5*eval0 + 0.5*eval1 ;

> I have implemented fractional plies for gnubg - in a slightly different
> way than the straight average used for both your tests and Joseph's. If
> you are compiling your own gnu, I'll be happy to send you the patch (the
> one I sent to the bug-gnubg earlier was broken). Unfortunately, it only
> does fractional plies above 1-ply :-( I think will look at implementing
> 0.5-ply soon.

How is your implementation different than straight average? 
 
> > -  Would it be possible to build a hybrid evalator into GnuBG like the
> >    one described above that would use a combination of zero-ply,
> >    one-ply, and 0.5-ply evaluation depending on the relative pipcount?
> 
> I think this is a little to specialized for what I would want to put
> into gnubg - at least until it has been more rigorously tested. 

Yes, you would want to have it well tested.  And, even before that,
there may be better ways that 0-ply and 1-ply can be usefully combined.
As I say, I'm willing to share my database of positions with anyone who 
has ideas they'd like to try out.

Tom