Re: [Bug-gnubg] race net: 0ply and 1ply

[Gary Wong]

On Wed, Nov 27, 2002 at 07:55:44AM +1300, Joseph Heled wrote:
> I now have a race net whose 0ply move error (as given by the race benchmark 
> which you kindly helped to roll) is 62% of the current net. The 2ply error 
> is 90% (i.e. 10% better). Alas, 1ply error is twice as large!
> 
> This is a somewhat bizarre result. I think I know in part why and will try 
> to solve this, but it seems that improving the 1ply will degrade the 0ply.
> 
> This raises a theoretical question. would YOU rather use a net with better 
> 0ply and 2ply, or a lesser one with a better 1ply?

Personally I would be at least a little bit suspicious of a net with
poor 1-ply performance (which is not to say that I wouldn't use it,
merely that I would be worried if I didn't have some sort of
explanation).

For instance, poor 1-ply performance is what we would expect to see
from a net with poor generalisation (which might occur if the training
data set is too close to the comparison data set).  In that case, the
"good" 0-ply (and to a lesser extent, 2-ply) performance might be largely
illusory.  (If a net gives excellent 0-ply results right across the
universe of inputs, it can't possibly give bad 1-ply results.)

This (and similar problems) make me uneasy about poor 1-ply performance.

Cheers,
Gary.
-- 
   Gary Wong           address@hidden           http://www.cs.arizona.edu/~gary/