gpsd-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [gpsd-dev] U-Blox M8, PPS and NTPsec - GPSd logic enhancement


From: Gary E. Miller
Subject: Re: [gpsd-dev] U-Blox M8, PPS and NTPsec - GPSd logic enhancement
Date: Mon, 8 Apr 2019 13:47:43 -0700

Yo Martin!

On Sun, 7 Apr 2019 22:41:29 -0400
Martin Boissonneault <address@hidden> wrote:

>    Before I continue, I'll get a bit in the context of this Pi, why
> GPSd and what I was trying to do since I got it in January 2019:

OK.
 
>    This Raspberry Pi 3B+ serves as an ADSB feeder to a few networks.
> It needed accurate time to help correlate MLAT messages, so I
> installed NTPd (Classic). I then read about NTPsec and switched. I
> had to compile from source tarball because I could not find the
> package in the Raspian repos.

For some definition of "accurate time".  Pretty typical usage for
a RasPi.  I also have some ADS-B receivers in my office.

>    I then decided to add a GNSSr to improve precision, this one:
> Uputronics - uBLOX MAX-M8Q Breakout for Active Antennas 
> <https://store.uputronics.com/index.php?route=product/product&path=60_64&product_id=84>

Yup.  Uputronics makes good stuff.  I have several of their products.

>    I interfaced it (over serial to /dev/ttyAMA0, PPS to gpio 18) to
> the Pi with a prototyping hat on which I installed a 3.3v 
> regulator+capacitors to power the u-Blox. Of course, some
> configuration was required on the Pi for this to work.

Pretty much what the clockmaker howto describes.

>    I have kernel PPS working and NTPsec can use it, but I wanted to
> add time information using GPSd for times when Internet is
> inaccessible. The GPSd package on the repos is 3.16.4, so I installed
> from the 3.18.1 tarball. Using the files in the systemd folder of the
> tarball, I set up automatic start of GPSd.

Also standard.

> My main references to get NTPsec/GPSd working where GPSD Time Service 
> HOWTO 

Yeah, also check the clockmaker howto.

> > # pstree -paul | fgrep gpsd  
> The result shows -n /dev/gps0 /dev/pps0

Good.

> >> I have a single GNSS attached to the Pi.  
> > USB, serial, i2c?  
> Serial over /dev/ttyAMA0 through a link named gps0, but I also made a 
> link named gpsd0. The /dev/pps0 also has a link named gpspps0.

gps0 is a magic name.  You change it, you lose the magic.

> >> Why SHM(1)? GPSd was started from the
> >> command line, not by systemctl.  
> > Mixing systemd and command lines ways to start gpsd is very hard.
> > The "pstree" above should help disentangle what is really running.  
> 
> At some point I needed to troubleshoot so I did run GPSd from 
> command-line, but only for the troubleshooting session. The normal 
> method is with systemd, and I use systemctl to stop/start GPSd. As I 
> write, systemd/systemctl is the ONLY method I use.

Unclear why you complicated you life by doing things two ways.  Good
that you picked one.

> >> If memory is right, it showed as
> >> SHM(1) instead of SHM(2). The fix was too easy to care as to why. I
> >> just added the SHM(1) for those times it does. It's disabled now.  
> > Except if you are randomly getting SHM(1) or SHM(2) then things are
> > not going well.  
> It's not random. Since I have exclusively used systemctl to
> start/stop GPSd, it has always been SHM 0 and 2.

Good, you have removed one source of randomness.  In that
configuration (-n /dev/gps0 /dev/pps0) the SHM(0) is serial time and
the SHM(2) is PPS time.

> > A common mistake to try to fix things in ntpd, but is it very
> > important that gpsd is working perfectly before starting ntpd.
> >
> > Always debug the first failure.  
> Agreed. Nothing worse than changing multiple things at a time. The 
> circle of confusion grows. Unfortunately, I got the GNSSr after I
> began using NTPd. And the combo has been giving me nice accuracy for
> the last few months.

Ain't broke, don't fix it.

> > Eventually you will remove the noselect and add a large offset so
> > it never gets selected.  But let's not get ahead of you first
> > failure. 
> >> It's the serial timestamp, not the PPS from GPSd?  
> > Correct.  The PPS does not tell you the date, just the top of the
> > seccond.  
> 
> I think I don't understand SHM(0) and SHM(2) properly then. I thought 
> that the SHMs carried accurate time as determined by GPSd from the 
> GNSSr's PPS and serial messages.

Sort of.  PPS tells you the top of the second, but not which second.

> So, SHM(2) carried the accurate time
> at the PPS  edge as determined from the GNSS time messages? In a way, 
> SHM(2) would be time calculated from serial messages but corrected to 
> the edge of PPS by GPSd? Hard to put in words. English is not my
> first language...

"would be", but not always.  Thus SHM(0)

> Or maybe the SHM timestamps are relative to the local timer, and NTPd 
> has to correlate all that into true time. I'll go read more about NTP 
> SHM. References welcome!

Has nothing to do with NTP SHM, this is in gpsd, before the time is
sent out (SHM, JSON, etc.).

> Don't think about SHM(1), I haven't seen it for many months. Not
> since I only use systemd/systemctl to start/stop NTPd and GPSd.

You brought up.  since you no longer see it, leave it for now.

> >> GPSD_SOCKET="/var/run/gpsd.sock"  
> > Since you are hard coding the GPS, and you are not doing hotplug,
> > then you do not want the soacket.  
> I'd be glad to disable that, but I haven't given time to learn how to
> do it.

Your pstree shows you already disabled that.

> >> --/usr/lib/systemd/system/gpsd.service:  
> > Also not part of gpsd, and unsupported here.  
> Taken from the systemd in the tarball. Manually installed.

Now you see why it is unsupported here.

> >> After=chronyd.service
> >> Before=ntpd.service  
> > Running chronyd and ntpd?  How do you manage that?  
> Absolutely not!! NTPsec only!!! The file came with Chronyd references.

Then remove them.

> >> # Give a few seconds for init, that way NTPd will have valid data
> >> on start ExecStart='/bin/sleep 10'  
> > That is a joke...  
> More my attempt to have GPSd ready and serving before starting NTPd. 
> Suggestions welcome!

a) no need to start gpsd before ntpd.
b) it can take up to 25 mins (12.5 * 2) before gpsd reports good time.

> >>>> Only one shows up, usually SHM(2).  
> >>> "usually" is not a good thing.  That shows a major problem.  Fix
> >>> that before continuing.  
> >> It shows only when I start GPSd from the command-line. Haven't seen
> >> it for many months because I am using systemctl to stop/start GPSD,
> >> and it is SHM(2) now. SHM(1) is a ghost from testing.  
> > Make up your mind: systemd or command line.  
> systemd. All NTPd/GPSd start/stop is handled by systemd. I start/stop 
> with systemctl when needed. NO other method is used to run them. The 
> only exception is command-line for debugging, and only for the 
> troubleshooting session.

But you just said you have not done that for months?  Pick one.

> >> sudo systemctl disable gpsd - to prevent interference
> >>
> >> sudo ubxtool -e BINARY -f /dev/gps0
> >> sudo ubxtool -d NMEA -f /dev/gps0  
> > Prolly best to swap those.  For a while you have both BINARY and
> > NMEA, at 9600, and that may very well cause buffers to overflow.
> >  
> >> sudo ubxtool -s 9600 -S 115200 -f /dev/gps0
> >> sudo ubxtool -s 115200 -f /dev/gps0  
> > Get the speed up, first.  To avoid buffer drop.  
> Done. I now set speed first and then reconfigure NMEA and UBX
> messages.

Done what?  Always best to just show your work.

> > Sort of.  Stop doing "sudo".  Just become root.  
> 
> I find it more convenient for scripts, and that was from one. Sure, I 
> could sudo <script>, but it had some issues when I tried with some of
> my past scripts. I don't remember the specifics, but this way of
> doing fixed my issues.

And created more.  "sudo" is a great big security hole.  If you don't care
to run a secure system then we don't care...

> As for becoming root, that changes my current folder. My scripts are 
> under a subfolder of my home folder, and I'd have to cd back to that 
> folder again and again. 

Yeah, but sudo is also not doing other things that people expect in root.
Thus mass confusion, in addition to the seecurity hole.

    Sure, putting all those scripts in the root
> home folder would fix that, but then I would not be able to modify
> them from WinSCP because of permissions. I could fix the permissions,
> but what is the point in the end?

You have a lot to learn about linux...

> man systemctl says it's from systemd?!?
> systemctl - Control the systemd system and service manager
> DESCRIPTION
>         systemctl may be used to introspect and control the state of
> the "systemd" system and service manager. Please refer to systemd(1) 
> for an
>         introduction into the basic concepts and functionality this
> tool manages.
> 
> What am I misunderstanding?

See above: we can't support systemd here.  Out of our control.

> >> Now, about /sudo ubxtool -d SBAS -f /dev/gps0/: it does NOT disable
> >> SBAS.  
> > Well, duh.  You just started systemctl, so /dev/gps0 is no longer
> > available.  
> 
> No, I mean running ubxtool -d SBAS -f /dev/gps0 BEFORE starting GPSd 
> does not change the status of SBAS in CFG-GNSS.

Once again, works for me.  And a lot of other people.

Send the output of this:

# killall gpsd; sleep 5; killell gpsd
# ubxtool -p GNSS -f /dev/gps0
# ubxtool -d SBAS -f /dev/gps0
# ubxtool -p GNSS -f /dev/gps0

Send ALL of the output, no edits.


> Confirmed from serial sniffing. I have no idea why, something to look 
> into later. Bug?

Sounds like what happens when gpsd is run from systemd
> 
> ubxtool: disable SBAS
> sent:
> UBX-CFG-GNSS:
>   Ver: 0 ChHw; 0 ChUse: ff, Blocks: 1
>   gnssId: SBAS TrkCh: 1 maxTrCh: 3, Flags: 0x1 01 00 00
>    L1C/A
> 
> I get $GPTXT,01,01,01,More than 100 frame errors, UART RX was 
> disabled*6E in the u-Center console sometimes.

Once again, looks like you have serial port issues.  Are you sure your
voltage levels on TX, RX, VCC and PPS are correct?  Noise on the line?
Noisy ground?

Did you remember to reset the serial port speed first?  ubxtool does not
do autobaud.


> Once GPSd runs, ubxtool -d SBAS 127.0.0.1:2947:/dev/gps0 DOES work. I 
> get ACK-ACK for 0x06-0x3E.

Also sounds like serial port speed issues.  Are you resetting the
serial port speed when gpsd is off?

> >> I'm trying to use known tools, but I still have to use u-Center to
> >> disable SBAS and set Timing-friendly settings.  
> > No, you do not need u-center.  But you gotta learn how to use
> > ubxtool correctly.  Let gpsd start first.  Let it autobaud,
> > configure, etc.  Then use ubxtool, without the -f, to configure the
> > GPS.  You have to order all wrong.  
> 
> GPSd will set it's own configuration on start, including enabling
> SBAS incorrectly (described below).

Once again, gpsd does NOT set SBAS, unless told to do so by ubxtool.

> Not good for timing, but them I'm
> not supposed to interfere.

The differences are so small you need a high quality OXCO to tell the
difference.  That is not your case.  Ignore the very small difference.

> I have to run ubxtool -d SBAS 
> 127.0.0.1:2947:/dev/gps0 at every start of GPSd now.

Once again, looks like a hardware error on your serial line.  Confirmed
by your error message above.

> The jumper
> prevents GPSd to do enable SBAS. I'm fixing GPSd with hardware.

Sounds like you are breaking it.

> GPSd
> resets the GNSSr to saved config on stop.

No.  gpsd sends nothing to the GPS on exit.

> So, I configure to my
> liking and save. It would be so nice if the required message list was
> easy to find or configurable.

I told you where to look before: driver_ubx.c line 1771 and beyond.

Or, as also told you man ytimes: use gpsctl or ubxtool.

> I know GPSd is meant to be transparent and return the configuration
> to what it was before it started,

Uh, no.  Where did you get that idea.

> and I normally would appreciate it. 
> Unfortunately, the default is not what I want outside of GPSd use. I 
> must configure it and save. u-Center makes it 100x easier than
> ubxtool.

Use what works for you.  Many people disagree.  Different strokes.

> Keep in mind that I want it up fast so NTPd has time available on
> start. Autobauding and reconfiguration is an annoyance in that
> context, but only in that context.

So preset the speed.  If you need time fast, then best to fetch the
time before starting gpsd and ntpd.

> It would be perfect if I could get the GNSS configured as GPSd
> expects it with speed preset BEFORE GPSd starts.

Easy, fix your serial port problems, then use ubstool to configure.
Many people do that without issues.

> No wasted time on
> autobauding and reconfiguring. Of course, GPSd should not reset it
> while exiting. That would be with -b?

-b is unrelated to autobaud.

> >> CFG-TP5: For 0-TIMEPULSE: Set Cable Delay to 15ns, that's for my
> >> antenna. CFG-CFG: Save current configuration.  
> > Why set NMEA version, then turn off NMEA?  Stop doing pointless
> > things...  
> Because I might enable some NMEA messages for use by other programs I 
> have on Windows.

gpsd does not support WindBlows.

> It is unused, but not pointless
> to me.

Except it messes up gpsd.

> > None of that will have a noticeable change in your ntp stability.
> >
> > Stop over complicating...  
> 
> Which part? Why? I'm listening.

See above.

> >>> Once again, gpsd did not do that, but you really do not want to
> >>> use Galileo and Glonass when doing timing.  So says the offical
> >>> u-blox doc.  
> >> Ghost in the machine:
> >> When the jumper is connected, SBAS gets reactivated.  
> > And you wonder why I keep saying don't do that?  
> 
> I don't want SBAS for timing. When the jumper is on, it turns on. No 
> jumper, no SBAS. And ubxtool -d SBAS -f /dev/gps0 does not turn it
> off.

Once again, works for everyone else.  Something broken in your system.

> GPSd _does_ turn it on on start. I logged the GPSd start and stop 
> commands sent.

Ah, someone slipped that into driver_ubx.c line 1551.  So, once again,
either use -b, or use ubxtool to adjust to your desire.

> In CFG-SBAS, the test bit must be OFF. Otherwise, information from
> SBAS sats in testing will be used. Here, it means that SBAS not for 
> production use will be used! Not good.

The SBAS in testing work just fine.  red herring.

> Take note that this byte string is not the best for everyone, like
> me. It sets SBAS in Autoscan mode. I can see one SBAS sat from
> Europe, but it is inappropriate for selection as it would not carry
> information for my geographic area.

The almanac is the same, everywhere in the world.  Trust your GPS to
only use the parts that help it.

> The recommended setting would
> include only the current WAAS svns, sv131, 135, 138. This setting
> should be configurable by the user.

Once again, overcomplicating.  If you can see the bird, you can use
the data.  If you can't see the bird, no problem.

> >> Arduino a few feet away sometimes spontaneously reboots, and it's
> >> connected to the same computer.  
> > I think you are over thinking it.  More likely a glitch when you
> > move the jumper causes a reset.  
> 
> GPSd *does *turn on SBAS on start. I captured the commands sent on 
> GPSd's start today. So, my jumper fixes the issue.

Yes, I see that now, but you still have a lot of problems, and from
your discription above, the cure is worse than the disease.

> >> For now, I'll leave the jumper OFF, as it's already configured.
> >> After that ghost, I disabled GPS/QZSS so they are the only ones
> >> enabled. SBAS disabled, config checked and saved.  
> > Note that u-blox doc says: don't do that!  GPS and QZSS should both
> > be on or both off.  
> Oops, I meant everything BUT GPS/QZSS. I wrote that part too late at 
> night and couldn't re-read it properly ;-)

Easy to do with ubxtool.  But, you'll find that u-blox does not use
the others if you have good GPS signals.  So, once again, you are
overcomplicating things.  Let the u-blox pick the optimal configuration.

> >>>> OK, then I must ask what does GPSd expect?  
> >>> I guess you keep missing the part where I say to set your GPS with
> >>> gpsctl or ubxtool.  
> >> Not every setting I need to set is easily accessible from those
> >> tools.  
> > Think about that.  Maybe it is not easy because it is the wrong
> > thing to do?  
> 
> Well, I disagree with that. By default, the GPSd+u-Blox M8
> combination is not optimal for timing purposes. What option do you
> offer to set antenna cable length delay.?

Your 15 ns is well under what you can meaure with a RasPi.  You are
adjusting thinigs well below the jitter level.  Pointless.

IFF you had a traceable UTC source, then you would use the fudge in
ntp.conf to remove the 15 ns.  But with your setup you'll never
see any change.

Remember, the smallest time increment the RasPi can measure is
52 ns.

From talking to people that do have traceability, 15 ns is the smaller
of your many error sources.

> Again, keep in mind that ubxtool -d SBAS -f /dev/gps0 is not
> disabling SBAS in my setup even if it says it does. I tried it with
> GPSd stopped, of course.

And, once again, keep in mind, no one else has that problem.  I suspect
you are forgetting to set the port speed before starting ubxtool.


> Likely a bug, but I have to work around it.

A bug on your end.

> >> I ran "ubxtool -e BINARY -f /dev/gps0" above. It enabled
> >> NAV-POSECEF, NAV-DOP, NAV-VELECEF, NAV-SBAS, plus those I already
> >> had, NAV-SOL, NAV-TIMEGPS. It prefers NAV-SVINFO to NAV-SAT, I
> >> have no problem with that.  
> > Before or after running gpsd?  
> Before.

And yet, you know that is failing for you.

> > If before you run gpsd, then the settings
> > are lost.  
> 
> Well, not quite, now that it is been set and saved. The configuration 
> does not change because I ran it, let it change it and injected my 
> adjustments with a save command. GPSd does send a reset command, but
> it only brings back the same configuration that I did save. It resets
> to same.

No point going around this merry go round again...

[...]

RGDS
GARY
---------------------------------------------------------------------------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
        address@hidden  Tel:+1 541 382 8588

            Veritas liberabit vos. -- Quid est veritas?
    "If you can’t measure it, you can’t improve it." - Lord Kelvin

Attachment: pgpmY1vWBObRs.pgp
Description: OpenPGP digital signature


reply via email to

[Prev in Thread] Current Thread [Next in Thread]