Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OverallResultsCases element to XML reporter #1958

Merged
merged 1 commit into from
Jun 28, 2020

Conversation

rpavlik
Copy link
Contributor

@rpavlik rpavlik commented Jun 11, 2020

Description

This adds a small additional output to the XML reporter: the number of passed/failed/expected-failed test cases and not just assertions. I am working with a team that is building somewhat of a system test, rather than unit test, with Catch2, and some assertions are called in a loop making the total assertion count not as helpful as the total test case count.

The approval tests appear to fail but for apparently unrelated reasons?

@horenmar horenmar added the Tweak label Jun 12, 2020
@horenmar
Copy link
Member

The approval tests fail for a reason that is very much related. They serve to check that the reporter output looks the way it should, which obviously does not match with the new changes. The reason the failure looks weird is that the checking script then attempts to print out diff of the incongruities, runs into a character it cannot print to the console and then dies.

@horenmar
Copy link
Member

The changes themselves look reasonable.

@rpavlik
Copy link
Contributor Author

rpavlik commented Jun 16, 2020

Ah, thanks for interpreting the test results for me. I'll see if I can add that change to the expected output.

@horenmar
Copy link
Member

To update the baselines, run scripts/approve.py and check that the resulting diff makes sense.

@horenmar horenmar force-pushed the xml-test-case-summary branch from b78c159 to fb32c83 Compare June 28, 2020 18:43
@codecov
Copy link

codecov bot commented Jun 28, 2020

Codecov Report

Merging #1958 into master will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1958      +/-   ##
==========================================
+ Coverage   88.59%   88.61%   +0.02%     
==========================================
  Files         138      138              
  Lines        5619     5627       +8     
==========================================
+ Hits         4978     4986       +8     
  Misses        641      641              

@horenmar horenmar merged commit ee4538c into catchorg:master Jun 28, 2020
@rpavlik rpavlik deleted the xml-test-case-summary branch July 7, 2020 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants