Re: [gpsd-dev] U-Blox M8, PPS and NTPsec - GPSd logic enhancement

[Gary E. Miller]

Yo Martin!

On Sat, 6 Apr 2019 02:02:50 -0400
Martin Boissonneault <address@hidden> wrote:

> > Once again, you have still not said where SHM(1) and SHM(2) come
> > from.  
> 
> Well, the only answer I have is that it's from GPSd.

OK, so what are you telling gpsd to do?  Here is how to
find the running gpsd command line, as root:

# pstree -paul | fgrep gpsd

> I have a single GNSS attached to the Pi.

USB, serial, i2c?

> Why SHM(1)? GPSd was started from the
> command line, not by systemctl.

Mixing systemd and command lines ways to start gpsd is very hard.
The "pstree" above should help disentangle what is really running.


> If memory is right, it showed as
> SHM(1) instead of SHM(2). The fix was too easy to care as to why. I
> just added the SHM(1) for those times it does. It's disabled now.

Except if you are randomly getting SHM(1) or SHM(2) then things are
not going well.

A common mistake to try to fix things in ntpd, but is it very 
important that gpsd is working perfectly before starting ntpd.

Always debug the first failure.

> Modified:
> #refclock shm unit 1 refid gPPS prefer
> 
> Right now, GPSd gives me SHM(0) and SHM(2) as visible from ntpshmmon.
> As it should.

No, SHM(2) says something bad is happening.

> >> # GPS Serial data reference (NTP0)
> >> #refclock shm unit 0 refid GNSS flag1 0 time1 0.0 minpoll 0
> >> maxpoll 8 stratum 10 noselect
> >> #refclock shm unit 0 refid GNSS flag1 0 time1 0.025 minpoll 0
> >> maxpoll 8 stratum 10 noselect
> >> # stratum 10  
> > Why is this commented out?  You need SHM(0) to get the right second
> > for PPS.  
> 
> I fail to understand how I need SHM(0) if I'm to make it /noselect/
> in ntp.conf?

You need it to know the GPS is working.

Eventually you will remove the noselect and add a large offset so
it never gets selected.  But let's not get ahead of you first failure.

> It's the serial timestamp, not the PPS from GPSd?

Correct.  The PPS does not tell you the date, just the top of the
seccond.

> "noselect: Marks the server as unused, except for display purposes.
> The server is discarded by the selection algorithm." 
> From:https://docs.ntpsec.org/latest/ntp_conf.html
> 
> Really confusing.

And display purposes will tell us more about your problem.  Later
we'll worry about selection.

> >>> what is conencted to SHM(2) and SHM(2)?  
> >> Both SHM(1) and SHM(2) are from GPSd.  
> > Obviously, but what is gpsd configured to send as SHM(1) and SHM(2)?
> > And what happend to SHM(0)?  Very strange configuration.  
> Depending on how I start GPSd, I get SHM(0) and (SHM(1) /or /SHM(2),
> not both). If systemctl starts GPSd, it's gonna be SHM(2). Anyway, I 
> commented out the SHM(1) refclock as it's not normally there.

You can't debug a random system.  Pick one, get it to work.

> >> They are the SHM related to the
> >> GNSS PPS from the offsets I see. Both are the same PPS from the
> >> GNSS attached.  
> > You have two PPS sources?  What does that look like on your gpsd
> > command line?  
> 
> Only one PPS source (GPIO pin), plus the serial port (no PPS on it).

But your PPS flips between SHM(1) and SHM(2).  Figure out why.

> --/etc/default/gpsd:
> # Default settings for the gpsd init script and the hotplug wrapper.

That file is NOT part of gpsd.  It is not supported here.  And it you
are starting manually, this file is not used at all.

So, pick one: use systemd, or use some other command line method.

> # Devices gpsd should collect to at boot time.
> # They need to be read/writeable, either by user gpsd or the group
> dialout. #DEVICES="/dev/ttyAMA0"
> DEVICES="/dev/gps0 /dev/pps0"

I assume you set /dev/gps0 as a link to /dev/ttyAMA0?

> # Other options you want to pass to gpsd
> GPSD_OPTIONS="-n"
> 
> GPSD_SOCKET="/var/run/gpsd.sock"

Since you are hard coding the GPS, and you are not doing hotplug, then
you do not want the soacket.

> --/usr/lib/systemd/system/gpsd.service:

Also not part of gpsd, and unsupported here.

> [Unit]
> Description=GPS (Global Positioning System) Daemon
> Requires=gpsd.socket
> # Needed with chrony SOCK refclock
> After=chronyd.service
> Before=ntpd.service

Running chronyd and ntpd?  How do you manage that?

> # Give a few seconds for init, that way NTPd will have valid data on
> start ExecStart='/bin/sleep 10'

That is a joke...

> >> Only one shows up, usually SHM(2).  
> > "usually" is not a good thing.  That shows a major problem.  Fix
> > that before continuing.  
> It shows only when I start GPSd from the command-line. Haven't seen
> it for many months because I am using systemctl to stop/start GPSD,
> and it is SHM(2) now. SHM(1) is a ghost from testing.

Make up your mind: systemd or command line.

> >> I also have SHM(0) from GPSd, and that one jumps ~100ms every 45
> >> minutes. I assume SHM(0) is from the serial messages. It's
> >> disabled.  
> > Don't disable that.  You need that to provide the proper second for
> > the PPS.  Just "noselect" it.  
> If I do so, how can NTPsec use it?

Have to confirm it is good, and how far off.  Then add a fudge so it
never gets selected, unless nothing else is available.  The remove the
unselect, and that helps steer the PPS to the right second.

But, one step at a time.

> "noselect: Marks the server as unused, except for display purposes.
> The server is discarded by the selection algorithm." 
> From:https://docs.ntpsec.org/latest/ntp_conf.html
> 
> Makes me really confused to "need" it, but having it "for display 
> purposes".

Don't try to over think it, small steps to where you need to go.

> All right, u-Blox has been reset to defaults with CFG-CFG, Revert to 
> default. I have everything backed up.
> 
> sudo systemctl disable gpsd - to prevent interference
> 
> sudo ubxtool -e BINARY -f /dev/gps0
> sudo ubxtool -d NMEA -f /dev/gps0

Prolly best to swap those.  For a while you have both BINARY and NMEA,
at 9600, and that may very well cause buffers to overflow.

> sudo ubxtool -s 9600 -S 115200 -f /dev/gps0
> sudo ubxtool -s 115200 -f /dev/gps0

Get the speed up, first.  To avoid buffer drop.

Sort of.  Stop doing "sudo".  Just become root.
 
> sudo systemctl enable gpsd

Which just undid all that work.  Either use systemd, or don't.  You can't
mix them.

> Now, about /sudo ubxtool -d SBAS -f /dev/gps0/: it does NOT disable
> SBAS.

Well, duh.  You just started systemctl, so /dev/gps0 is no longer
available.

> I'm trying to use known tools, but I still have to use u-Center to 
> disable SBAS and set Timing-friendly settings.

No, you do not need u-center.  But you gotta learn how to use
ubxtool correctly.  Let gpsd start first.  Let it autobaud, configure,
etc.  Then use ubxtool, without the -f, to configure the GPS.  You have
to order all wrong.


> Now, I set:
> CFG-ANT: Set according the the hardware.
> CFG-GNSS: Disable SBAS, Enable Galileo. Sets CFG-SBAS to Disabled.
> (Read to the end, GPS/QZSS are the only enabled.)
> CFG-ITFM: Set antenna to "active", Enabled.
> CFG-NAV5: Dynamic: Stationary, Mode: 2D, set HAE to that of my
> location. CFG-NMEA: NMEA version 4.1, Numbering 1-Extended. (Doesn't
> matter, the messages have been turned off.)
> CFG-PMS: Full Power. Sets CFG-RXM Power to 0-Continous.
> CFG-PRT: For 1-UART1: Protocol out 0+1 UBX+NMEA.NMEA messages are off 
> anyway.
> CFG-TP5: For 0-TIMEPULSE: Set Cable Delay to 15ns, that's for my
> antenna. CFG-CFG: Save current configuration.

Why set NMEA version, then turn off NMEA?  Stop doing pointless things...

None of that will have a noticeable change in your ntp stability.

Stop over complicating...

> > Once again, gpsd did not do that, but you really do not want to use
> > Galileo and Glonass when doing timing.  So says the offical u-blox
> > doc.  
> Ghost in the machine:
> When the jumper is connected, SBAS gets reactivated.

And you wonder why I keep saying don't do that?

> I had disabled the ability for GPSd to send to my GNSSr and it fixed
> the problem, but I now see GPSd is not at fault. Constellations get 
> disabled, although indirectly. It's very odd that only SBAS is
> affected. could be a u-Blox bug. Dang. I now need to find out why...
> Possibly some ground loop or RF, my Arduino a few feet away sometimes
> spontaneously reboots, and it's connected to the same computer.

I think you are over thinking it.  More likely a glitch when you move the
jumper causes a reset.

> For now, I'll leave the jumper OFF, as it's already configured. After 
> that ghost, I disabled GPS/QZSS so they are the only ones enabled.
> SBAS disabled, config checked and saved.

Note that u-blox doc says: don't do that!  GPS and QZSS should both be
on or both off.

> >> OK, then I must ask what does GPSd expect?  
> > I guess you keep missing the part where I say to set your GPS with
> > gpsctl or ubxtool.  
> 
> Not every setting I need to set is easily accessible from those
> tools.

Think about that.  Maybe it is not easy because it is the wrong thing
to do?

> Besides, the tool does not matter, as long as the
> configuration is good. That is why I described my changes above..

Which do not look good...

> I ran "ubxtool -e BINARY -f /dev/gps0" above. It enabled NAV-POSECEF, 
> NAV-DOP, NAV-VELECEF, NAV-SBAS, plus those I already had, NAV-SOL, 
> NAV-TIMEGPS. It prefers NAV-SVINFO to NAV-SAT, I have no problem with
> that.

Before or after running gpsd?  If before you run gpsd, then the settings
are lost.  If after you run gpsd then /dev/gps0 is unavailable.

Either way, nul operations.

> >>> We can't really support configurations that are not well tested
> >>> and part of our regressions.  
> >> I don't mind giving full GNSS control back to GPSd, but then, why
> >> was it disabling Galileo and Glonass?  
> > No, it was not.  And anyway that is the right thing to do.  
> No, GPSd does not send the commands. I even recorded the stream to 
> check. It somehow gets "disabled", but not by GPSd. That is something 
> somewhat solved. :-/

If gpsd not sending commands, then something is badly broken.

> >>> Too early to look at ntpmon, your pool servers not running yet.  
> >> Well, they where up, and after a while, they where released. NTPd
> >> picks the best and drops the unneeded extras?  
> > No.  
> When NTPd starts, it does use a associate to a bunch of pool servers. 
> They are later pruned by NTPsec as needed. I just use the pool 
> statement. I don't mess with anything else. If they get pruned, I
> have no hand in it. That fits what I read there: 
> https://docs.ntpsec.org/latest/discover.html

I think you misunderstand what it is saying, but low priority.

> >> SHM(1)          .gPPS.           0 l  64    0 0ns      0ns
> >> 954ns  
> > Why is SHM(1) broken?  
> 
> Already answered: It's there only for when GPSd is started from the 
> command-line while debugging. Wierd, but it works, and then it goes
> back to SHM(2) when systemctl starts it. Disabled now anyway.

Which tells me that when you run those two ways you get very different
results.  That flip-flopping is confusing you.

RGDS
GARY
---------------------------------------------------------------------------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
        address@hidden  Tel:+1 541 382 8588

            Veritas liberabit vos. -- Quid est veritas?
    "If you can’t measure it, you can’t improve it." - Lord Kelvin

pgpBcL6mDjwU6.pgp
Description: OpenPGP digital signature