Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It takes minutes for the report portal to display test results even though a corresponding test suite completes in 30 seconds #40

Open
AlexThreeSixty opened this issue Jan 7, 2025 · 3 comments
Labels
help wanted Extra attention is needed

Comments

@AlexThreeSixty
Copy link

Describe the bug
Actual test execution finishes in ~30 seconds but reportportal represents it as minutes long

Steps to Reproduce
Steps to reproduce the behavior:

  1. Run tests (i.e. mvn clean test "-Dkarate.options=--tags ${{ github.event.inputs.tags }}" -Dkarate.env=${{ github.event.inputs.env }} - we run from GitHub pipelines)
  2. Observe the test execution in the report portal

Expected behavior
Report portal shows "near real time" test execution results

Actual behavior
Report portal shows test execution results with minutes long delays. The whole suite finishes in ~30 while reportportal only starts showing first results at around that time. Then several minutes later (4-5) it shows all the results and the suite as completed, "duration" at the "all launches" page as those 4-5 minutes.
Then refreshing the "all launches" page changes the previously 4-5 minutes duration to the real ~30 seconds

Dependency versions
com.epam.reportportal:client-java:jar:5.2.23 (also tried with 5.1.2)
com.epam.reportportal:agent-java-karate:jar:5.1.2

Additional context
It's used to work fast (almost real-time reflecting the test results) but, apparently, we were running from a forked-reportportal-implementation rather than master (for example, agent-java-karate groupId was com.github.vrymar)
The issue started appearing after [vrymar] has migrated our repo to the reportportal main branch

here are some screenshots
Whole test suite pipeline finishes within ~1:30 (there are 104 test cases + extra setup/teardown steps)
ActualTestSuiteFinishesInSeconds

While reportportal even minutes later still shows that tests are being executed and only partially completed
ReportPortalShowsTestSuiteAsBeingExecutedInMinutes

Opening a test confirms that: no test results are there in the reportportal yet
ReportPortalShowsTestAsBeingExecuted
ReportPortalStillNoResults

By the time reporportal finally finishes loading results, it shows that the execution time was 4-5 minutes
ReportPortalShowsFirstActualTimeThatItTookToLoadTheResults

But refreshing the page reveals a reality: the tests were completed within ~30 seconds
AfterARefreshReportPortalMagicallyShowsARealExecutionTime

@AlexThreeSixty AlexThreeSixty added the bug Something isn't working label Jan 7, 2025
@HardNorth
Copy link
Member

HardNorth commented Jan 17, 2025

@AlexThreeSixty, the fact that your job completes within seconds tells clearly, that this is not the Agent issue. If Agent would have any performance / upload issues it will just hold your job until it finish. So the real issue is how your server manage to process the data Agent sent. Certainly, the Agent started to send more data after new implementation, just check how the internals of your tests are look like. But this is not a bug, it's improvement to organize views and utilize our analyzer service.

For further investigation please provide server and component versions and at least common numbers of internal items and logs you have in these tests to estimate the load you put on backend.

BTW: do not change client-java library version manually. It's our common code library which might not contain necessary functionality if you downgrade it.

@AlexThreeSixty
Copy link
Author

hi @HardNorth
Thanks for replying. May I ask for some clarifications, please, to be able to provide answers:

  1. what do you mean by "your server"? (in the statement "So the real issue is how your server manage to process the data Agent sent. Certainly"). Is that where reportportal is deployed? (if yes, I see "Build: 5.11.0")
  2. "component version" - what component is meant here?
  3. "common numbers of internal items" - is that how many scenarios? (if yes, 107 scenarios within 22 tests)
  4. "logs" - size on a disk? (if yes, 2.6 MB raw file and 270kb zipped)

@HardNorth HardNorth added help wanted Extra attention is needed and removed bug Something isn't working labels Jan 21, 2025
@rakesh123822
Copy link

rakesh123822 commented Jan 31, 2025

I can confirm the exact issue occurring in my project as well. After adding hook for Reportportal in my runner it takes time to finish a test run before starting the next test run. It almost seems that the test is stuck.

Edit 1: I noticed this error when my test scenario is calling @setup scenario. Here's the exception that i get

10:48:36.706 [pool-4-thread-1] ERROR c.e.r.karate.ReportPortalHook - ERROR: Trying to post unspecified feature.
10:48:36.720 [pool-4-thread-1] ERROR c.e.r.karate.ReportPortalHook - ERROR: Trying to finish unspecified scenario.
10:48:36.721 [pool-4-thread-1] ERROR c.epam.reportportal.service.Launch - ItemID should not be null
java.lang.NullPointerException: ItemID should not be null

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants