bug#23521: XFAIL

[Mathieu Lirzin]

Hi,

Reuben Thomas <address@hidden> writes:

> The documentation says: "It's not uncommon, especially during early
> development stages, that some tests fail for known reasons, and that
> the developer doesn't want
> to tackle these failures immediately (this is especially true when the
> failing tests deal with corner cases)."
>
> Another common use for "expected failure" is to write tests to check
> that error conditions arise as expected, for example, by checking that
> a program raises an error when given invalid input.

I agree that XFAIL can be ambiguous, however I think this usage is not
desirable.  It gives an additional opposite meaning to XFAIL symbol
which makes it even more confusing.

> If that's a reasonable use of automake's test harness, perhaps the
> documentation could reflect that, e.g. by adding:
>
> "Another use for XFAIL is to mark tests that are supposed to fail, for
> example, to check that a program raises an error when given invalid
> input."
>
> It is often easier to write expected-to-fail tests this way (so that
> they can all look the same), rather than have to have, for example, an
> extra driver that converts expected errors into success codes for the
> automake test harness.

What do you mean precisely by “an extra driver”?

Thanks

-- 
Mathieu Lirzin