bug#16057: Non-parallel test suite API changes in 1.13

[Stefano Lattarini]

On 12/05/2013 10:49 PM, Eric Blake wrote:
> On 12/05/2013 03:28 PM, Stefano Lattarini wrote:
>> tags 16057 notabug
>> close 16057
>> stop
>>
>> On 12/05/2013 01:37 AM, Behdad Esfahbod wrote:
>>> Hi,
>>>
>> Hello Behdad.
>>
>>> Please advise how one is supposed to port their pre-1.13 test suite's
>>> TESTS_ENVIRONMENT to 1.13.  Currently, in fontconfig and harfbuzz at least, 
>>> we
>>> cannot find a solution that works both with 1.11 and 1.13, and we cannot
>>> require 1.13.
>>>
>> Why not? 1.13 is almost one year old now...
> 
> Believe it or not, some people still like to make their project work
> with out-of-the-box tools.
>
I believe it, but I disagree with this approach, when the out-of-the-box
tools are too outdated (But see below, the bit about documentation).

> In the case of libvirt, we strive to make
> our Makefile.am work on the software available in CentOS 5 (hello,
> patched automake 1.9) - and it is not a coincidence that this is also
> the oldest version still actively supported by gnulib.  Just because
> newer releases are now a year old, and in spite of the fact that
> automake is fairly easy to self-install from a tarball, does NOT imply
> that everyone is willing to self-install.
>
And I'm not willing to support such unwillingness, unless it's very
easy do.  Sorry.

As I've already admitted, in retrospective, switching the default
testsuite behaviour in a minor release have been a bad idea, since I
broke the expectations of compatibility held by the users.  But I
can't change that mistake now now, and I believe the best way
forward from the situation we are in is to encourage developers to
update their tools.  (And I believe we should do so also in the
future -- but only by bumping major versions more frequently, not
by breaking compatibility between minor versions).

> For good or for bad, there
> are enough developers that think of the autotools as magic black boxes
> that they are unwilling to use any version not shipped by their distro.
>
I'm not willing to go to great lengths to support such attitude in the
developers, sorry.

> So, anything that upstream automake can do to document HOW to write a
> Makefile.am that works on both modern AND ancient automake is appreciated.
> 
OTOH, I see no harm in documenting possible workarounds; we already do so
so for other use cases we don't really support (e.g., keeping autotools
generated files in a Git repository): it's easy to do, doesn't require
us to maintain more code, and makes it clear to the user that it is not
an encouraged setup and that maintaining it is his responsibility, not
ours.  So, patches welcome.

>> You might be able to keep 1.11 compatibility with some more-or-less
>> hacky workaround, but I'm not going to try to work out the details,
>> because I believe that is the wrong approach.  Automake is only
>> required by developers, it's easy to install (and to install multiple
>> versions in parallel, if different level of APIs compatibilities are
>> needed), and only requires a very tiny amount of disk space.
>>
>> I'm closing this bug as "works as intended".  Sorry.
> 
> I'm sorry to see this attitude; it is in the best interest as a
> developer tool to specifically document how to write code that still
> works with older versions.  While it is okay to encourage the use of
> newer tools, we can't be so blind as to assume that people won't (or
> even can't) upgrade as fast as we'd like.
>
We don't completely disagree in this at least (see above).

> Is AM_INIT_AUTOMAKE in 1.11 smart enough to handle m4 conditionals in
> the set of options requested?
>
I think so (I'm not going to try it myself, but if somebody else does,
I won't throw away the insight he gets ;-)

> And if so, can that be coupled with the
> use of m4_ifdef or some other m4 code to probe for a feature that only
> exists in 1.13 or newer?  If so, try something along the lines of:
> 
> AM_INIT_AUTOMAKE([1.11.1 gnits ...]m4_ifdef([1.13-witness], [ serial-tests])
> 
The problem with this might be if the "1.13-witness" gets screwd up
by distro-specific patches somehow.  If you manage to avoid that, seems
a reasonable workaround.

Regards,
  Stefano