[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [gpsd-dev] âCEP(50) 0.009 meters
From: |
Gary E. Miller |
Subject: |
Re: [gpsd-dev] âCEP(50) 0.009 meters |
Date: |
Mon, 25 Jul 2016 17:54:42 -0700 |
Yo Hal!
On Mon, 25 Jul 2016 17:50:40 -0700
Hal Murray <address@hidden> wrote:
> address@hidden said:
> > 0.0000001 degree at the equator is 11.132 mm.
> > At centimeter accuracy my latest code at the equator is off by
> > almost 40% in some test cases...
>
> From a few messages ago:
> > 121.31405412
> > - 121.31405411
> > --------------
> > 0.00000001
>
> You are likely to have serious roundoff in the decimal printing.
Works for me: %.9f
# python test_misc.py
misc test: 0.00000000 0.00000000, 0.00000000 0.00000010, expected 0.011320 got
0.011057
misc test: 67.00000000 0.00000000, 67.00000000 0.00000010, expected 0.004350
got 0.004358
So I'm under 1% now in lat. Still fighting lon...
RGDS
GARY
---------------------------------------------------------------------------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
address@hidden Tel:+1 541 382 8588
pgpXri9pefJoG.pgp
Description: OpenPGP digital signature