Re: Invalid position given by Telit HE910-EUG

[Titouan Christophe]

Hello Gary and all,

On 4/04/20 21:14, Gary E. Miller wrote:
> Yo Titouan!
>
> On Sat, 4 Apr 2020 12:37:12 +0200
> Titouan Christophe <address@hidden> wrote:
>
> > > Ouch.  And you are sure it is plain NMEA0183?
> >
> > As far as I can tell, yes.
>
> Not totally convincing.

If I open a serial console on the GPS port, all I get is plain NMEA0183. 
I also checked with hexdump that there are no undisplayable characters.

> > > How did you ruin gpspipe?  With "-R", "-r", or "-w"?
> >
> > Some more context:
>
> I usually prefer my simple questions are answered before further
> elaboration.  So the answer is??
>
> > we're remotely operating a fleet of embedded
> > devices that are installed into rail vehicles.
>
> Not sure why that matters to this issue.

Just wanted to let you know the context, how I obtained the data.

> > The device on which
> > the error happened is running gpsd 3.18.
>
> 3.18 is old.  Can you duplicate on 3.20 or from the git head code?
>
> > Our end user applications
> > use the gpsd C client, and makes use of
> > gpsdata->fix.latitude/longitude (amongst others).
>
> In that case, what I sent you proves thatt git head does the right thing.
>
> > This is how we
> > connect to gpsd and read TPV data:
> >
> > gps_stream(&gps_data, WATCH_ENABLE | WATCH_JSON | WATCH_NMEA, NULL);
> > while (running) {
> >       if (gps_waiting(&gps_data, 2000000)) {
> >           r = gps_read(&gps_data, NULL, 0);
> >           if (r > 0){
> >               // do something with gps_data
> >           }
> >       }
> > }
>
> Of course the issue you report is in the "do something" part.  So that
> snippet not useful.


Maybe the flags passed to gps_stream could have been wrong ?

As I stated in my original email, we use the content of 
gps_data->fix.latitude/.longitude/.speed/...

We then pack them into JSON format (with libjansson), and send them via 
MQTT. Here is an extract of our program:
http://paste.awesom.eu/fHqV

> > Because we have had a lot of trouble with these Telit HE910-EUG
> > chips,
>
> Yeah, they are junk.
>
> > we also run a daemon that continuously dump the NMEA stream
> > into a weekly rotated log, as to make post-mortem analysis of GPS
> > failures.
>
> Fair enough.
>
> > We invoke gpspipe like this:
> > gpspipe -r -T %s -n $LINES -o $FILENAME
>
> So dumping the raw NMEA.  Why not -R?  There may be stuff other than
> NMEA in there.

Well, the manpage is not very explicit about that. What's the difference 
between "raw" and "super raw" ?

> > > > As you can see, this GPS has also rolled over in year 2000, but
> > > > this is a different subject.
> > >
> > > gpsd in git head can often handle GPS week roll over (WKRO) problems
> > > now.
> >
> > This Telit chip has a pretty bad behaviour (see
> > https://rutronik-tec.com/wp-content/uploads/2019/02/Telit_Cellular_WeekRollover_Application_Note_r0.pdf),
>
> All you had to say was "Telit'>
>
> > and there was no WKRO patch available for 3.18 at that time (it was
> > maybe even 3.17 back then), so we maintain a patch out of the gpsd
> > tree for that matter; see http://paste.awesom.eu/QGyb
>
> Fixed in git head, so old news to this project.  My previous message
> showed you it works.
>
> So, back to your original issue.  The current gpsd code works for me.
> We agree the NMEA 0183 in is bad.  I showed you that current gpsd
> does the right thing with your data.  You have not shown that the
> gpsd 3.18 is sending bad data.

Sorry for losing your time then. There is no mention of such filtering 
in the release notes 
(https://gitlab.com/gpsd/gpsd/-/blob/master/NEWS#L19-47), so how could I 
have known ? My mail was intended to know if gpsd was supposed to handle 
that case or not.

I replayed the entire dump file with gpsdecode on both release-3.20 and 
the HEAD branch on my laptop; and indeed I do not observe the wrong 
position anymore, and the rollover is correctly handled.

For the record, when using gpsdecode from release-3.19 on the same dump 
file, I also obtain a wrong latitude:

{
  "class":"TPV",
  "device":"stdin",
  "mode":3,
  "time":"2020-04-03T12:02:13.953Z",
  "ept":0.005,
  "lat":153.588700000,
  "lon":7.975805000,
  "alt":211.000,
  "epx":8.085,
  "epy":10.408,
  "epv":34.500,
  "track":269.5800,
  "speed":0.005,
  "climb":0.000,
  "epd":0.0001,
  "eps":20.82,
  "epc":69.00,
  "eph":14.250,
  "sep":33.250
}

> Simple solution, update your gpsd version.


I will give a try to update gpsd to 3.20 in Buildroot 
(https://git.buildroot.org/buildroot/tree/package/gpsd) so we can update 
our fleet soon. However, it does not seem easy, because gpsd 3.20 does 
not cross-compile anymore as is.

The problem is in the SContruct file, in the function CheckSizeOf(), 
which compiles an executable (on the host building machine) using the 
cross-compiler, and then tries to execute it. Needless to say this 
cannot work when compiling an ARM executable on an x86_64 machine:
[...]
Checking for C header file termios.h... yes
Checking for C header file winsock2.h... no
Checking size of time_t ... sh: .sconf_temp/conftest_26: cannot execute 
binary file: Exec format error
no
sizeof(time_t) is 0
WARNING: time_t is too small.  It will fail in 2038
Checking for C function cfmakeraw()... yes
Checking for C function clock_gettime()... yes
Checking for C function daemon()... yes
[...]

I will try to find a workaround for that.



In the meantime I updated the is_valid() function in the snippet I 
linked earlier in this message, to only accept [-90, data->fix.latitude, 
90] and [-180, data->fix.longitude, 180] like this:

static inline int is_valid(const struct gps_data_t *data)
{
    return (data->status == STATUS_FIX) && (data->satellites_used >= 3) &&
           (data->fix.mode == MODE_2D || data->fix.mode == MODE_3D) &&
           (! isnan(data->fix.latitude)) &&
           (! isnan(data->fix.longitude)) &&
           (-90 <= data->fix.latitude && data->fix.latitude <= 90) &&
           (-180 <= data->fix.longitude && data->fix.longitude <= 180);
}

> RGDS
> GARY
> ---------------------------------------------------------------------------
> Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
>         address@hidden  Tel:+1 541 382 8588
>
>             Veritas liberabit vos. -- Quid est veritas?
>      "If you can't measure it, you can't improve it." - Lord Kelvin

Thanks for your time anyway.

Best regards,

Titouan