[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Excluding FAILs from UNSUPPORTED test cases
From: |
Jonathan Wakely |
Subject: |
Re: Excluding FAILs from UNSUPPORTED test cases |
Date: |
Fri, 23 Sep 2022 11:02:56 +0100 |
On Fri, 23 Sept 2022 at 10:54, Arsen Arsenović wrote:
>
> Hi,
>
> On Friday, 23 September 2022 05:33:42 CEST Jacob Bachmeyer wrote:
> > I have not yet had the occasion to dive into the dg.exp code, but a
> > quick look at the documentation (such as it is) in the header comment
> > block suggests that you need code in your "libstdc++-dg-prune"
> > procedure to recognize the unsupported cases and return the
> > "::unsupported::freestanding" string.
> Jonathan implemented this solution in the meanwhile, it did work as
> expected, though, it also produced more PASSes than anticipated, by
> going from:
> FAIL: 30_threads/promise/cons/assign_neg.cc (test for errors, line 29)
> UNSUPPORTED: 30_threads/promise/cons/assign_neg.cc: hosted C++ headers
> not supported
>
> ... to:
> PASS: 30_threads/promise/cons/assign_neg.cc (test for errors, line 29)
> UNSUPPORTED: 30_threads/promise/cons/assign_neg.cc: hosted C++ headers
> not supported
>
> Which is (IMO) good enough, though, I can't quite tell why that
> happens.
Yes, this is a little surprising. If a test is UNSUPPORTED because it
fails to match a target selector, then any dg-error or other
directives in the test are ignored, and there are no PASS, XFAIL etc.
lines for them in the results.
But when it's unsupported because ${tool}-dg-prune returns
"::unsupported::" the individual checks within the file still appear
in the results. I realise that in the target selector case, the test
is just skipped before doing anything, and in the prune case we don't
know that it's unsupported until *after* testing it. But it seems to
me that ideally the individual checks would get "retroactively
skipped" if tool-dg-prune returns one of ::untested::, ::unresolved::,
or ::unsupported::. Maybe that's not easy to do though.
- Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/22
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/22
- Re: Excluding FAILs from UNSUPPORTED test cases, Jonathan Wakely, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases,
Jonathan Wakely <=
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/24
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/24
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/26
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/29
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/30
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/30