Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix xml reporter crashing when a benchmark failed. #1706

Merged
merged 1 commit into from
Jul 29, 2019
Merged

Fix xml reporter crashing when a benchmark failed. #1706

merged 1 commit into from
Jul 29, 2019

Conversation

fodinabor
Copy link
Contributor

Description

When running a benchmark, that throws an exception or a optimized away exception is thrown, the xml reporter closes the wrong tag.
This was fixed by opening a BenchmarkResults tag on the preparing message. This way it is also reproducible, which benchmark failed.

@horenmar
Copy link
Member

So did it crash or did it write out wrong xml?

@fodinabor
Copy link
Contributor Author

fodinabor commented Jul 29, 2019

Both ;-)
It crashes because it tries to pop more tags than are opened..

<?xml version="1.0" encoding="UTF-8"?>
<Catch name="test-gpgpuevaluation-performance.exe" filters="[polarization][copy]">
  <Group name="test-gpgpuevaluation-performance.exe">
    <TestCase name="CPU Memory Transfers - Types&lt;cl_uchar, cl_uchar3, cl_uchar4>" tags="[copy][cpu][polarization]" filename="D:\Dokumente\Uni\Bachelorarbeit\Project\test\performance\src\test_cpu_performance.cpp" line="241">
      <Section name="Given: CPU Provider" filename="D:\Dokumente\Uni\Bachelorarbeit\Project\test\performance\src\test_cpu_performance.cpp" line="247">
        <Info>
          work_group_size := 1
        </Info>
        <Section name="And given: real polarization image" filename="D:\Dokumente\Uni\Bachelorarbeit\Project\test\performance\src\test_cpu_performance.cpp" line="256">
          <Info>
            work_group_size := 1
          </Info>
          <Expression success="true" type="REQUIRE" filename="D:\Dokumente\Uni\Bachelorarbeit\Project\test\performance\src\test_cpu_performance.cpp" line="262">
            <Original>
              data
            </Original>
            <Expanded>
              0x00000262a4f2e040
            </Expanded>
          </Expression>
          <failed message="could not measure benchmark, maybe it was optimized away"/>
        </Section>
        <failed message="could not measure benchmark, maybe it was optimized away"/>
      </Section>
      <failed message="could not measure benchmark, maybe it was optimized away"/>
    </TestCase>
    <OverallResults successes="1" failures="0" expectedFailures="0"/>
  </Group>
  <OverallResults successes="2" failures="0" expectedFailures="0"/>
</Catch>
<OverallResult success="true"/>
</>

@horenmar
Copy link
Member

Ok, I see.

@codecov
Copy link

codecov bot commented Jul 29, 2019

Codecov Report

Merging #1706 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1706      +/-   ##
==========================================
- Coverage    86.2%   86.18%   -0.02%     
==========================================
  Files         136      136              
  Lines        5174     5180       +6     
==========================================
+ Hits         4460     4464       +4     
- Misses        714      716       +2

1 similar comment
@codecov
Copy link

codecov bot commented Jul 29, 2019

Codecov Report

Merging #1706 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1706      +/-   ##
==========================================
- Coverage    86.2%   86.18%   -0.02%     
==========================================
  Files         136      136              
  Lines        5174     5180       +6     
==========================================
+ Hits         4460     4464       +4     
- Misses        714      716       +2

@horenmar horenmar merged commit 7e7ab0e into catchorg:master Jul 29, 2019
@fodinabor fodinabor deleted the failingBenchmarkXML branch July 29, 2019 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants