Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DependencyLibrary to support test case/suite dependency #1323

Merged
merged 2 commits into from
Mar 27, 2024

Conversation

manosnoam
Copy link
Contributor

Robot Framework DependencyLibrary allows creating dependency between test cases and test suites.
For example, we can create the foloowing logic:
If test A must be executed before test B, because test B expects some resources created by test A,
but test A failed - then test B will not be executed and marked as skipped.

Adopting DependencyLibrary Keywards often in our sequential tests can save a lot of run-time and diagnostic time, in case of failed tests that were dependent on previous tests.

This commit includes a first use of the KW Depends On Test in: 420__model_serving.robot suite.

image

Robot Framework DependencyLibrary allows creating dependency between
test cases and test suites. For example, if test A must be executed
before test B (because B expect some resources created by A),
but test A failed - then test B will not be executed.

Using DependencyLibrary often in our sequential tests can save run and
diagnostic time due to failed tests that were dependent on others.

This commit includes a first use of `Depends On Test` KW in:
420__model_serving.robot suite.

Signed-off-by: manosnoam <[email protected]>
Copy link
Contributor

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass %
447 0 0 447 100

Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

Copy link
Member

@jstourac jstourac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not in favor of having the dependent tests in general (setup and teardown seem nicer to me), but I agree that with our current situation and that setup may be slow for our use case, this can help. Thank you.

Also, there are some major upgrades of python dependencies included with this, so hopefully all will work as expected. Would be nice to run a couple of more tests (smoke?) with this PR too, though.

@tarukumar
Copy link
Contributor

Agree with Jan on running the smoke to make sure nothing is broken

@manosnoam
Copy link
Contributor Author

manosnoam commented Mar 27, 2024

I'm not in favor of having the dependent tests in general (setup and teardown seem nicer to me),

Counting on test setup (which means call test teardown later too) brings more code + more test runtime + more maintenance + debug time on falied tests that counts on earlier tests, as in my example:
Test Inference Without Token Authentication which runs in smoke after Verify Openvino_IR Model Via UI is expecting to have the model interface already running. Currently if you try to run Verify Openvino_IR Model Via UI solely, it will always fail. This is where DependencyLibrary is so effective.

Once we start to adopt it in our code, which anyhow uses a lot of dependency - without specifying it in the tests sometimes - we would save much more setup code lines, and failure analysis time.

Also, there are some major upgrades of python dependencies included with this, so hopefully all will work as expected. Would be nice to run a couple of more tests (smoke?) with this PR too, though.

Yes, running as we speak :)

@manosnoam manosnoam added verified This PR has been tested with Jenkins misc Miscelaneus (PR will be listed in release-notes) labels Mar 27, 2024
@manosnoam manosnoam merged commit a3de25b into red-hat-data-services:master Mar 27, 2024
11 of 12 checks passed
@manosnoam
Copy link
Contributor Author

I verified on 2.9 and the the pass rate was similar as we had for 2.9 before this commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
misc Miscelaneus (PR will be listed in release-notes) verified This PR has been tested with Jenkins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants