Re: [gpsd-dev] GPSD and the Python 3 transition

[Fred Wright]

On Sat, 26 Mar 2016, Eric S. Raymond wrote:

> Fred Wright <address@hidden>:
> > Things like regression tests, OTOH, should be run with the target Python,
> > for maximum consistency with the intended target environment.
>
> Good rule in general.  In GPSD's specific application context I'd be
> willing to let this slip if it bought a large enough reduction in the
> build or test framework complexity, because of long experience that
> Python version-skew problems are never - ever - related to or revealing of
> problems in the GPSD code itself.

Agreed, though once one has the concept of "target Python" vs. "scons
Python", it's just a matter of using the right prefix in each context.

> > If the version of Python where the libraries were installed is removed,
> > then the libraries were removed with it, and so the install is broken no
> > matter what you do.  But changing the default Python *without* removing
> > the one where the libraries were installed shouldn't break the install,
> > which is what happens with the generic shebang.
>
> To fix this, we'd have to at minimum (a) templatize 7 files (gegps,
> gpscat, gpsfake, gpsprof, test_maidenhead.py, xgps, and xgpsspeed).

It's not full templatization for the specific case of shebang replacement.
It's fairly simple Python code to copy a file with a replacement first
line.  That means that the "source" would be valid Python code as is.
Based on that, I'd be inclined to just have foobar.py as the "source", and
a builder that derives foobar from foobar.py by replacing the shebang line
with something that may or may not be the generic version that it already
has.

BTW, test_maidenhead.py doesn't belong on that list, since it's not an
*installed* program, and already gets run with an explicit Python version
(currently the scons Python, but that could be fixed).

> Then we'd have to add hair to the scons recipe to plug in a specified
> Python version as the value of the @PYTHON@ cookie.

Just a builder as noted above.  The value would be derived in some fashion
from an option.

> I'm resistant to this because (a) the build recipe is cryptic enough
> already, and (b) I don't see a big win here.  I know you do because
> you're thinking in terms of deliberately having multiple Pythons
> installed, but that's a tiny corner case.  Real users never deliberately
> change that version, and real users count on their *package system* to
> keep GPSD's library synced with the current default version.

As I mentioned before, I think this is going to become a lot more common
as Python 3 proliferates, and most users wind up having at least two
versions of Python installed.  Full support for Python 3 has to include
removing "highlander assumptions". :-)

> Thus, I'm willing to dump the problem on the packaging system.
> They already have to deal with this for every single project in the world
> that installs Python libraries anyway - that is, declare that a Python
> upgrade forces an upgrade to dependent packages.

I agree that it's up to the packaging system to decide what to put in
shebang lines, but it reduces maintainability to require the packaging
system to know *what to patch*.  And switching Python versions (e.g. by
changing the symlink) isn't necessarily an "upgrade", and shouldn't
necessarily demand a reinstall.

> > If a packaging system decides to support multiple Python versioning by the
> > brute-force method of installing multiple copies of the libraries, then it
> > would also be useful to have an install target for just those, so that it
> > could iterate over Python versions without duplicating installs of the C
> > code (or worse, uninstalling the C code if just one Python version were
> > uninstalled).
>
> Again, I think you're trying to take over a a job best left to the
> package system.  I'm unwilling to pay complexity cost for that.

And again, it reduces maintainability to require a packaging system
wanting to perform a partial install to have to ignore the entire install
mechanism and roll its own, now needing inside knowledge of what should be
installed.  Better to break up "install" into sub-installs, with the
"install" target being an alias for the full set.

This is largely about the modularity philosophy of trying to keep
knowledge contained where it belongs.

Fred Wright