Different results in the reports

[diegolovison]

TestNG Version

6.14.3

Description

We have 3 tests:

• FooTest:a
• FooTest:b
• BarTest:a

We have the following configuration in pom.xml

<excludedGroups>windows</excludedGroups>

In this case, 2 tests are supposed to be running and 1 should be skipped.

In the console we have:

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.176 sec - in TestSuite

In JUnitXMLReporter ( https://github.com/diegolovison/testng-reporter/blob/master/target/surefire-reports/Surefire%20suite/Surefire%20test.xml ) we have:

ignored="1"
<testcase name="a" ignored

In JUnitReportReporter ( https://github.com/diegolovison/testng-reporter/blob/master/target/surefire-reports/junitreports/TEST-com.github.diegolovison.testngreporter.FooTest.xml ) we have:

skipped="0"

No mention about FooTest:a

Expected behavior

I would like to have the same result in console, JUnitXMLReporter and JUnitReportReporter

Actual behavior

Different value in all reports

Is the issue reproductible on runner?

• Shell
• [ X ] Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

https://github.com/diegolovison/testng-reporter

[krmahadevan]

@diegolovison - Can you please check if this library that I built works for you ? https://github.com/RationaleEmotions/junitreport

[diegolovison]

Hi @krmahadevan

Well, no. I generated the report as you can see here https://github.com/diegolovison/testng-reporter/blob/rationale-emotions/target/surefire-reports/Surefire%20suite/Surefire%20test.xml
it does not show what I would like to see

The "junitreports" https://github.com/diegolovison/testng-reporter/tree/rationale-emotions/target/surefire-reports/junitreports shows what I am expecting.

What is your opinion?

[krmahadevan]

@diegolovison - I didn't quite understand. So are you saying that junitreports library that I shared works and solves your purpose?

[diegolovison]

@krmahadevan, no. In your report, where I can see that the method c in FooTest skipped?

  <testcase name="c" classname="com.github.diegolovison.testngreporter.FooTest">
    <skipped/>
  </testcase>

[krmahadevan]

@diegolovison - I am not sure if skipped is a recognized status in the JUnit world. My library refers to the JUnit.xsd available here https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd

[diegolovison]

Well, the info above related with FooTest:c comes from JUnitReportReporter as you can see here: https://github.com/diegolovison/testng-reporter/blob/rationale-emotions/target/surefire-reports/junitreports/TEST-com.github.diegolovison.testngreporter.FooTest.xml

What I would like to address in this issue is keep console, JUnitXMLReporter and JUnitReportReporter consistent. Today they have different results what cause confusion.

[krmahadevan]

@diegolovison - Let me see what I can do. Do you think you could help quickly share a sample that can be used to reproduce the problem ? That way its easy to get to the bottom of this issue.

[diegolovison]

@krmahadevan yes. It is described on the issue :)
#1830 (comment)
There is a test case here https://github.com/diegolovison/testng-reporter
If you have any question, please let me know