bug#22471: ert batch should print compile parsable error messages

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> Currently, when running in batch, ert prints out messages like so:
>>
>> Running 24 tests (2016-01-27 09:05:17+0000)
>>    passed   1/24  buffer-string=
>>    passed   2/24  buffer=
>> ...
>>    passed  22/24  sisyphus-test-with-find-file
>>    passed  23/24  to-string
>>    passed  24/24  with-temp-buffers
>>
>> Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)
>>
>> 1 unexpected results:
>>    FAILED  crash-out
>>
>> It would be nice to add some compilation-mode parsable data to this. So,
>> something like
>>
>>    passed   1/24  buffer-string=  (in ./test.sisyphus-test.el:22:)
>
> I think that would look rather cluttered for the "passed" lines, but it
> would indeed be helpful on the FAILED lines.

This would in my opinion be very useful and a great timesaver.  But yes,
for FAILED results only, much like how the compiler only gives us line
numbers if there is a problem.  (AFAIR, that's how the test frameworks
I've used do it.)

So I updated the patch from the feature/parsable-ert-output against
current master (patch attached), made sure that one of our tests failed,
and then tested it with "make check".

I got the following backtrace, but gave up after looking at it for a
while:

Debugger entered--Lisp error: (error "Don’t know where
‘fns-tests-string-bytes’ is defin...")
  signal(error ("Don’t know where ‘fns-tests-string-bytes’ is defin..."))
  error("Don't know where `%s' is defined" fns-tests-string-bytes)
  find-function-search-for-symbol(fns-tests-string-bytes ert-deftest nil)
  find-definition-noselect(fns-tests-string-bytes ert-deftest)
  ert-test-location([...])
[...]

0001-Add-source-information-to-ERT-batch-mode.patch
Description: Text Data