Re: [gpsd-dev] test failure

[Eric S. Raymond]

Roy Barkas <address@hidden>:
> The following sequence of operations resulted in a test failure testing a
> build of  git head 3.10~dev (revision release-3.9-843-gd4a915c) :
> 
>  
> 
> 1.       Cloned the head
> 
> 2.       Ran scons
> 
> 3.       Ran scons -c (because I had forgotten to load dependencies
> 
> 4.       Ran sudo apt-get build-dep gpsd
> 
> 5.       Ran scons
> 
> 6.       Ran scons check
> 
>  
> 
> The test failed as below  ( I then temoved the gpsd source directory,
> recloned it and tried again without the intervening scons -c and build-deps
> - the tests failed again although with fewer errors.

This has all the earmarks of your test delays not being large enough.
The nondeterminism is kind of the smoking gun on that.  Contemplate
this code from gps/fake.py:

# The two magic numbers below have to be derived from observation.  If
# they're too high you'll slow the tests down a lot.  If they're too low
# you'll get random spurious regression failures that usually look
# like lines missing from the end of the test output relative to the
# check file.  The need for them may be symptomatic of race conditions
# in the pty layer or elsewhere.

# WRITE_PAD: Define a per-line delay on writes so we won't spam the
# buffers in the pty layer or gpsd itself. Values smaller than the
# system timer tick don't make any difference here.

# CLOSE_DELAY: We delay briefly after a GPS source is exhausted before
# removing it.  This should give its subscribers time to get gpsd's
# response before we call the cleanup code. Note that using fractional
# seconds in CLOSE_DELAY may have no effect; Python time.time()
# returns a float value, but it is not guaranteed by Python that the C
# implementation underneath will return with precision finer than 1
# second. (Linux and *BSD return full precision.)

# Field reports:
#
# Eric Raymond  on Linux 3.11.0 under an Intel Core Duo at 2.66GHz.
#  WRITE_PAD = 0.0 / CLOSE_DELAY = 0.1    Works, 112s real
#  WRITE_PAD = 0.0 / CLOSE_DELAY = 0.05   Fails
#
# Michael Tatarinov on a Raspberry Pi:
#  WRITE_PAD = 0.0 / CLOSE_DELAY = 0.05    Works, 344s real
#  WRITE_PAD = 0.0 / CLOSE_DELAY = 0.0     Fails, 339s real
#
# From Hal Murray on NetBSD 6.1.2 on an Intel(R) Celeron(R) CPU 2.80GHz
#  WRITE_PAD = 0.0 / CLOSE_DELAY = 0.4    Works, takes 688.69s real
#  WRITE_PAD = 0.0 / CLOSE_DELAY = 0.3    Fails tcp-torture.log, 677.53s real
#
# Greg Troxel running NetBSD 6 on a core i5 (i386, 4 cpus) 2.90GHz.
#  WRITE_PAD = 0.001 / CLOSE_DELAY = 0.2 had failures (645s)
#  WRITE_PAD = 0.001 / CLOSE_DELAY = 0.4 had failures (662s)
#  WRITE_PAD = 0.004 / CLOSE_DELAY = 0.8 all tests passed
#  WRITE_PAD = 0.001 / CLOSE_DELAY = 0.8 all tests passed (697s)
#

if sys.platform.startswith("linux"):
    WRITE_PAD = 0.0
    CLOSE_DELAY = 0.1
elif sys.platform.startswith("freebsd"):
    WRITE_PAD = 0.001
    CLOSE_DELAY = 0.4
else:
    WRITE_PAD = 0.004
    CLOSE_DELAY = 0.8

Try making these larger until your tests stop flaking out.  Then report
those values with distinguishing information about your platform, please.
-- 
                <a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>