Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TestNG fails with versions 7.6 and up #427

Closed
lprimak opened this issue Sep 14, 2022 · 2 comments · Fixed by #440
Closed

TestNG fails with versions 7.6 and up #427

lprimak opened this issue Sep 14, 2022 · 2 comments · Fixed by #440

Comments

@lprimak
Copy link
Contributor

lprimak commented Sep 14, 2022 •
edited
Loading

When upgrading to versions of 7.6 or higher of TestNG, Arquillian stops working

Reproducer: https://github.com/flowlogix/test-arq-suite/tree/TestNG

  • update pom.xml to use the latest testng (7.6 or higher)
    mvn verify (TestContainers-based, docker required)
    mvn verify -Ppayara-server-managed (will locally download and run Payara-based tests)

Exception:

[ERROR] Tests run: 6, Failures: 5, Errors: 0, Skipped: 0, Time elapsed: 5.342 s <<< FAILURE! - in TestSuite
[ERROR] com.flowlogix.arqsuite.ContainerStartTest.dummy  Time elapsed: 0.452 s  <<< FAILURE!
org.testng.TestNotInvokedException: 

com.flowlogix.arqsuite.ContainerStartTest.dummy defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:695)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:806)
	at org.testng.TestRunner.run(TestRunner.java:601)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
	at org.testng.TestNG.runSuites(TestNG.java:1099)
	at org.testng.TestNG.run(TestNG.java:1067)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:151)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:192)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:93)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
[ERROR] com.flowlogix.arqsuite.DeploymentOneTest.one  Time elapsed: 0.032 s  <<< FAILURE!
org.testng.TestNotInvokedException: 

com.flowlogix.arqsuite.DeploymentOneTest.one defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:695)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:806)
	at org.testng.TestRunner.run(TestRunner.java:601)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
	at org.testng.TestNG.runSuites(TestNG.java:1099)
	at org.testng.TestNG.run(TestNG.java:1067)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:151)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:192)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:93)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
[ERROR] com.flowlogix.arqsuite.DeploymentOneTest.two  Time elapsed: 0.042 s  <<< FAILURE!
org.testng.TestNotInvokedException: 

com.flowlogix.arqsuite.DeploymentOneTest.two defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:695)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:806)
	at org.testng.TestRunner.run(TestRunner.java:601)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
	at org.testng.TestNG.runSuites(TestNG.java:1099)
	at org.testng.TestNG.run(TestNG.java:1067)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:151)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:192)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:93)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
[ERROR] com.flowlogix.arqsuite.DeploymentTwoTest.one  Time elapsed: 0.047 s  <<< FAILURE!
org.testng.TestNotInvokedException: 

com.flowlogix.arqsuite.DeploymentTwoTest.one defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:695)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:806)
	at org.testng.TestRunner.run(TestRunner.java:601)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
	at org.testng.TestNG.runSuites(TestNG.java:1099)
	at org.testng.TestNG.run(TestNG.java:1067)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:151)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:192)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:93)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
[ERROR] com.flowlogix.arqsuite.DeploymentTwoTest.two  Time elapsed: 0.029 s  <<< FAILURE!
org.testng.TestNotInvokedException: 

com.flowlogix.arqsuite.DeploymentTwoTest.two defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:695)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:806)
	at org.testng.TestRunner.run(TestRunner.java:601)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
	at org.testng.TestNG.runSuites(TestNG.java:1099)
	at org.testng.TestNG.run(TestNG.java:1067)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:151)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:192)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:93)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   ContainerStartTest.dummy » TestNotInvoked 
com.flowlogix.arqsuite.ContainerStartTest.dummy defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
[ERROR]   DeploymentOneTest.one » TestNotInvoked 
com.flowlogix.arqsuite.DeploymentOneTest.one defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
[ERROR]   DeploymentOneTest.two » TestNotInvoked 
com.flowlogix.arqsuite.DeploymentOneTest.two defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
[ERROR]   DeploymentTwoTest.one » TestNotInvoked 
com.flowlogix.arqsuite.DeploymentTwoTest.one defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
[ERROR]   DeploymentTwoTest.two » TestNotInvoked 
com.flowlogix.arqsuite.DeploymentTwoTest.two defines a callback via org.testng.IHookable but neither the callback was invoked nor the status was altered to SUCCESS|FAILURE|SKIP|SUCCESS_PERCENTAGE_FAILURE
[INFO] 
[ERROR] Tests run: 6, Failures: 5, Errors: 0, Skipped: 0
@lprimak
Copy link
Contributor Author

lprimak commented Oct 28, 2022

This will surely affect MicroProfile TCK at some point @Emily-Jiang

@Emily-Jiang
Copy link

This will surely affect MicroProfile TCK at some point @Emily-Jiang

Thanks for tagging me! Yes, this will affect MicroProfile TCK for sure.

@lprimak lprimak mentioned this issue Oct 28, 2022
Merged
3 tasks
@lprimak lprimak mentioned this issue Nov 18, 2022
Closed
lprimak added a commit to flowlogix/arquillian-core that referenced this issue Nov 18, 2022
@lprimak
fixes failures with TestNG 7.6.1+
f22b7b5 
fixes arquillian#427
This was referenced Nov 18, 2022
Closed
Closed
Merged
bartoszmajsak pushed a commit that referenced this issue Nov 19, 2022
@lprimak
fix: propagates failures with TestNG 7.6.1+ (#440)
9028d37 
Fixes #427
fixes flowlogix/test-arq-suite#49
@tevans78 tevans78 mentioned this issue Jan 5, 2023
Closed
@pdudits pdudits mentioned this issue Feb 8, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@lprimak @Emily-Jiang