Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testNG symbol for Pipeline syntax clarity #155

Merged
merged 1 commit into from
Sep 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,39 @@ Results**. This option allows you to configure the following properties:

### Pipeline in Jenkinsfile

The link:https://www.jenkins.io/redirect/pipeline-snippet-generator[Pipeline Syntax Snippet Generator] guides the user to select TestNG report options.
Add the `testNG` step to declarative Pipeline in a `post` section.

```
post {
always {
step([$class: 'Publisher', reportFilenamePattern: '**/testng-results.xml'])
testNG()
}
}
```

Additional options can be included in the testNG declarative Pipeline step like this:

```
post {
always {
testNG(showFailedBuilds: true,
unstableFails: 5, unstableSkips: 25,
failedFails: 10, failedSkips: 50)
}
}
```

The `testNG` Pipeline step can be used in a scripted Pipeline like this:

```
node {
// Add steps that run TestNG tests
// Publish TestNG report with the `testNG()` step
testNG(reportFilenamePattern: '**/testng-many-results.xml')
}
```

### Properties

Some TestNG plugin properties can only be controlled by command line properties set at Jenkins startup.
Expand Down
2 changes: 2 additions & 0 deletions src/main/java/hudson/plugins/testng/Publisher.java
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import hudson.util.FormValidation;
import jenkins.tasks.SimpleBuildStep;
import net.sf.json.JSONObject;
import org.jenkinsci.Symbol;
import org.kohsuke.stapler.*;
import org.kohsuke.stapler.verb.POST;

Expand Down Expand Up @@ -442,6 +443,7 @@ static boolean saveReports(FilePath testngDir, FilePath[] paths, PrintStream log
return true;
}

@Symbol("testNG")
public static final class DescriptorImpl extends BuildStepDescriptor<hudson.tasks.Publisher> {

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,29 @@ public void test_threshold_for_fails_default_pipeline() throws Exception {
r.assertLogContains("tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE", build);
}

@Issue("JENKINS-27121")
@Test
public void test_threshold_for_fails_default_pipeline_using_symbol() throws Exception {
if (isWindows()) {
/* Fails to delete a file on Windows agents of ci.jenkins.io.
* Likely indicates a bug somewhere, but I'd rather have most
* of the tests passing on ci.jenkins.io Windows rather than
* blocking all Windows tests until this can be investigated.
*/
return;
}
WorkflowJob p = r.jenkins.createProject(WorkflowJob.class, "p");
String contents = CommonUtil.getContents(Constants.TESTNG_FAILED_TEST);
p.setDefinition(new CpsFlowDefinition("node {\n writeFile(file: 'testng-results.xml', text: '''" + contents + "''')\n testNG()\n}\n", true));
WorkflowRun build = p.scheduleBuild2(0).get();
r.assertBuildStatus(Result.UNSTABLE, build);
TestNGTestResultBuildAction action = build.getAction(TestNGTestResultBuildAction.class);
assertNotNull(action);
TestNGResult result = action.getResult();
assertEquals("checking result details", "TestNGResult {totalTests=2, failedTests=1, skippedTests=0, failedConfigs=0, skippedConfigs=0}", result.toString());
r.assertLogContains("tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE", build);
}

@Test
public void test_threshold_for_fails_failure() throws Exception {
FreeStyleProject p = r.createFreeStyleProject();
Expand Down