Save results from each 'specDone' to json file and read&send these results to TestRail

[viktorgogulenko]

Hey, guys. As seems you already know about an issue with sending results in Protractor with "shardTestFiles: true" and seems Protractor guys are not going to fix it in nearest future, I would kindly ask you for an improvement which could make possible to use your "jasmine-2-testrail" also with enabled sharding. I've looked into the code of your module and found that in 'specDone' with enabled sharding we are still getting results and pushing them into 'this.results' array, but at the end of tests run Protractor is destroying this array and your module can't send anything as 'results' array is empty. Error message is:

Created new test run: AIMSAutomatedTestRun 5/25/2020, 9:05:42 AM
Field :results cannot be empty (one result is required)

I've added little bit more logging and seeing that each spec has results, but this results are not existent at the end:

[09:06:01] I/testLogger - [chrome #1-0] PID: 7490
[chrome #1-0] Specs: Test1.spec
[chrome #1-0]
[chrome #1-0] https://selenoid.clouddev.safetyio.com:5443/wd/hub/
[chrome #1-0] [09:05:43] I/local - Starting selenium standalone server...
[chrome #1-0] [09:05:45] I/local - Selenium standalone server started at http://192.168.178.23:63545/wd/hub
[chrome #1-0] Started
[chrome #1-0]
[chrome #1-0] Executing 1 defined specs...
[chrome #1-0]
[chrome #1-0] Test Suites & Specs:
[chrome #1-0]
[chrome #1-0] 1. Test 1
[chrome #1-0] Going to page: https://test1.com
[chrome #1-0] . ✔ 183689: Test 1 (11702ms)[ { case_id: 183689,
[chrome #1-0] status_id: 1,
[chrome #1-0] comment: 'Test passed successfully.' } ]
[chrome #1-0]
[chrome #1-0]
[chrome #1-0]
[chrome #1-0] 1 spec, 0 failures
[chrome #1-0] Finished in 11.72 seconds
[chrome #1-0]
[chrome #1-0]
[chrome #1-0]
[chrome #1-0] >> Done!
[chrome #1-0]
[chrome #1-0]
[chrome #1-0] Summary:
[chrome #1-0]
[chrome #1-0] Suites: 1 of 1
[chrome #1-0] Specs: 1 of 1
[chrome #1-0] Expects: 2 (0 failures)
[chrome #1-0] Finished in 11.719 seconds
[chrome #1-0]
[chrome #1-0] [09:06:01] I/local - Shutting down selenium standalone server.
[09:06:01] I/testLogger -
[09:06:01] I/launcher - 1 instance(s) of WebDriver still running
.[09:06:07] I/testLogger -
[09:06:07] I/testLogger - [chrome #1-1] PID: 7491
[chrome #1-1] Specs: Test2.spec
[chrome #1-1]
[chrome #1-1] https://selenoid.clouddev.safetyio.com:5443/wd/hub/
[chrome #1-1] [09:05:43] I/local - Starting selenium standalone server...
[chrome #1-1] [09:05:45] I/local - Selenium standalone server started at http://192.168.178.23:50453/wd/hub
[chrome #1-1] Started
[chrome #1-1]
[chrome #1-1] Executing 1 defined specs...
[chrome #1-1]
[chrome #1-1] Test Suites & Specs:
[chrome #1-1]
[chrome #1-1] 1. Test2
[chrome #1-1] Going to page: https://test2.com
[chrome #1-1] . ✔ 194045: Test 2 (16883ms)[ { case_id: 194045,
[chrome #1-1] status_id: 1,
[chrome #1-1] comment: 'Test passed successfully.' } ]
[chrome #1-1]
[chrome #1-1]
[chrome #1-1]
[chrome #1-1] 1 spec, 0 failures
[chrome #1-1] Finished in 16.893 seconds
[chrome #1-1]
[chrome #1-1]
[chrome #1-1]
[chrome #1-1] >> Done!
[chrome #1-1]
[chrome #1-1]
[chrome #1-1] Summary:
[chrome #1-1]
[chrome #1-1] Suites: 1 of 1
[chrome #1-1] Specs: 1 of 1
[chrome #1-1] Expects: 3 (0 failures)
[chrome #1-1] Finished in 16.892 seconds
[chrome #1-1]
[chrome #1-1] [09:06:07] I/local - Shutting down selenium standalone server.
[09:06:07] I/testLogger -
[09:06:07] I/launcher - 0 instance(s) of WebDriver still running
[09:06:07] I/launcher - chrome #1-0 passed
[09:06:07] I/launcher - chrome #1-1 passed
[]
Created new test run: AutomatedTestRun 5/25/2020, 9:05:42 AM
Field :results cannot be empty (one result is required)

Logging which I've added:

case 'passed':
          this.results.push({
            case_id: parseInt(id, 10),
            status_id: 1,
            comment: 'Test passed successfully.',
          })
          console.log(this.results);
          break;

if(runName){
      name = runName
    }
    console.log(this.results);
    api.addRun(projectId, {
      suite_id: suiteId, name: name, include_all: false, case_ids: this.caseids,
    }).then((r) => {
      console.log('Created new test run: ' + name)
      api.addResultsForCases(r.id, { results: this.results })
        .then(() => {
          console.log('Added test results')
        })
        .catch((error) => { console.log(error.message || error) })
    }).catch((error) => { console.log(error.message || error) }
    )

May I ask you to make an improvement in your module instead of saving in memory to save results into json file (with possibility to re-write specific result if was used 'retry-spec' action), read it at the end of tests run and send these results to TestRail. As seems it could be a good workaround with sharding issue in Protractor and jasmine-console-reporter. Thanks a lot in advance!

[viktorgogulenko]

Guys, any thoughts, ideas or updates here?