[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Excluding FAILs from UNSUPPORTED test cases
From: |
Arsen Arsenović |
Subject: |
Re: Excluding FAILs from UNSUPPORTED test cases |
Date: |
Sat, 24 Sep 2022 17:07:49 +0200 |
Hi,
On Saturday, 24 September 2022 06:24:56 CEST you wrote:
> > But when it's unsupported because ${tool}-dg-prune returns
> > "::unsupported::" the individual checks within the file still appear
> > in the results. I realise that in the target selector case, the test
> > is just skipped before doing anything, and in the prune case we
> > don't
> > know that it's unsupported until *after* testing it. But it seems to
> > me that ideally the individual checks would get "retroactively
> > skipped" if tool-dg-prune returns one of ::untested::,
> > ::unresolved::, or ::unsupported::. Maybe that's not easy to do
> > though.
>
> This actually looks like a bug in dg.exp to me. The output from
> ${tool}-dg-test is first collected, all at once, (line 696) and then
> analyzed (lines 701-749). I would expect ${tool}-dg-prune to have an
> opportunity to modify the text to be analyzed /before/ the analysis is
> done, but the current code checks for messages first, /then/ calls
> ${tool}-dg-prune (and the framework's prune_warnings procedure) to
> remove irrelevant output, then declares excess errors if the final
> text is nonempty.
This is how I understood the output analysis routine: remove expected
lines first (i.e. those matched by dg-messages), then lines that
shouldn't be a FAIL but are noise or such instead (i.e. prune), which
allows ${tool}-dg-prune and dg-prune-output to be less careful about
what they remove (as they don't have to worry about removing diagnostics
that should be matched by dg-{error,warning}).
Changing ${tool}-dg-prune to execute earlier might be a bit too drastic,
because it could expose some overzealous pruning routines. On the other
hand, giving ${tool}-dg-test a way to terminate analysis early, before
all the output processing, could maybe constrained enough to not change
what existing code does (which is why I suggested it earlier).
What do you think?
Thanks,
--
Arsen Arsenović
signature.asc
Description: This is a digitally signed message part.
- Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/22
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/22
- Re: Excluding FAILs from UNSUPPORTED test cases, Jonathan Wakely, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Jonathan Wakely, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/24
- Re: Excluding FAILs from UNSUPPORTED test cases,
Arsen Arsenović <=
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/26
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/29
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/30
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/30