Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update libcxx expected_results.txt, skipped_tests.txt #812

Merged
merged 1 commit into from
May 12, 2020

Conversation

StephanTLavavej
Copy link
Member

  • except.nested/rethrow_if_nested.pass.cpp crashes on my machine, displaying a dialog box that isn't subject to the AssertionDialogAvoider. Accordingly, it should be SKIPPED instead of marked as known FAIL.
  • tuple.apply/apply_large_arity.pass.cpp actually compiles on my new machine, but it consumes an astounding 18 GB of memory. This should continue to be SKIPPED, but I'm updating the recorded reason.
  • basic.ios.members/move.pass.cpp is also crashing for me, so change FAIL to SKIPPED.
  • The tests that need aligned_alloc() are reliably failing to compile, and it's possible that this will be implemented in the next hundred years, so change SKIPPED to FAIL.
  • Update the explanation and bug citation for <cmath>: signbit() misses overloads for integer types #519 (the longstanding issue about integral overloads of <cmath> classification functions).
  • The system_error tests assuming POSIX error codes are failing reliably. Change SKIPPED to FAIL so if they're fixed upstream, we'll notice.
  • container.requirements.general/allocator_move.pass.cpp is also asserting reliably; change SKIPPED to FAIL.
  • Update the comments in skipped_tests.txt to match.

I checked with @CaseyCarter about the rationales for all other SKIPPED tests. They are:

  • ios.base.storage/iword.pass.cpp etc. running 50M loops takes forever, so we want to skip that.
  • futures.task.members/dtor.pass.cpp triggers undefined behavior, so we should skip potential crashes.
  • directory_entry.cons/copy.pass.cpp and the other <filesystem> tests might have side effects; we haven't analyzed them to see if they're safe to run (instead of, for example, recursively deleting directory trees that they shouldn't).
  • thread.condition.condvarany/wait_terminates.sh.cpp needs to be run specially; there's no point in trying to run it otherwise.
  • MEOW.verify.cpp tests need clang-verify; our infrastructure doesn't do that.
  • The tests for C++20 P0722R3 are SKIPPED so we can run the tests with the current development compiler in addition to the public VS Preview; this will vanish in the near future when VS 2019 16.7 Preview 1 is available.
  • The _Exit bug blocking thread.thread.assign/move2.pass.cpp etc. leads to runtime misbehavior which is best SKIPPED.
  • Bogus timing assumptions are bogus.
  • deque.modifiers/insert_iter_iter.pass.cpp timeouts (not yet analyzed) indicate that the test is taking forever, which should be SKIPPED until we understand the root cause.

@StephanTLavavej StephanTLavavej added the test Related to test code label May 8, 2020
@StephanTLavavej StephanTLavavej requested a review from a team as a code owner May 8, 2020 03:21
@CaseyCarter CaseyCarter self-assigned this May 12, 2020
@CaseyCarter CaseyCarter merged commit 738d9bf into microsoft:master May 12, 2020
@CaseyCarter
Copy link
Contributor

Thanks for fine-tuning our handling of libc++ tests! 🐱

@StephanTLavavej StephanTLavavej deleted the libcxx_skips branch May 13, 2020 02:18
@CaseyCarter CaseyCarter removed their assignment Jun 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Related to test code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants