Make DS Project tests independent #978

bdattoma · 2023-10-13T16:36:30Z

This PR wants to address the issue of having the DS Project tests dependent on each other which makes difficult moving the tests in and out of a Quality Gate.

How
The way to achieve some level of indendence implemented by this PR is the following:

having one common DS project created as part of Suite Setup
every test which needs a resource, must create it as part of test setup and delete it in the test teardown
- this was to me a bit concerning bcos it would increase the total duration of testing (a workbench takes around 40-50s to be up and running). However I tried mitigating this issue by merging some smaller test cases together.
- time impact is going to be assessed as part of the PR testing

The PR required many changes and I'll try to test as much as I can the impacted Suites

Merged test cases

ODS-1817,ODS-1815,ODS-1813,ODS-1784
ODS-1823, ODS-1818
The merge happened only at code repo level, not in Polarion: the new cumulative test case reports all the aggregated test IDs as RF Tags

Signed-off-by: bdattoma <[email protected]>

github-advanced-security

Robocop found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

github-actions · 2023-10-13T16:40:05Z

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %
366	0	0	366	100

Signed-off-by: bdattoma <[email protected]>

bdattoma · 2023-10-16T08:39:26Z

PR Validation:

rhods-ci-pr-test/2039 DS project Suite SUCCESS (23 minutes vs 28 minutes without this PR). "Edit" suite FAILED -> fixed and will re-test
- re-test edit suite after fixes: rhods-ci-pr-test/2043 SUCCESS (3 minutes vs 1 minutes without this PR)
rhods-ci-pr-test/2044 DS project Suite excluding Smoke tag and (Auto/Product)Bugs: SUCCESS (12 minutes)
rhods-ci-pr-test DS project Suite Smoke only SUCCESS (10 minutes vs 7 minutes)
rhods-ci-pr-test/2047 DS project additional suite excluding Resource-GPU tag SUCCESS (17 minutes vs 9 minutes)
rhods-ci-pr-test/2048/ Smoke (42 minutes vs 34 minutes). 4 Failures, but not related to this PR.
tested single test cases in isolation SUCCESS - I didn't do it via jenkins for sake of time, quite many test cases to check.

NOTE: dryrun failure is not caused by the code in this PR but by Embedded Operator test. We can fix in a separate PR, it's not critical

Signed-off-by: bdattoma <[email protected]>

FedeAlonso

There are a bunch of fixes (indentations, RETURN, etc) that can be easily fixed following the alerts.
LGTM

sonarqubecloud · 2023-10-20T07:53:25Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

jgarciao

There is some commented code that could be removed in this or a follow-up PR. Otherwise LGTM

bdattoma · 2023-10-23T09:03:25Z

There is some commented code that could be removed in this or a follow-up PR. Otherwise LGTM

yeah, sorry for that. I originally left some of them bcos I wasn't sure if they were still needed for test stability. I will remove commented in code in a next PR

* remove ODS-1812 Signed-off-by: bdattoma <[email protected]> * stop workbench from starting status Signed-off-by: bdattoma <[email protected]> * add kw to delete single project from cli Signed-off-by: bdattoma <[email protected]> * first attempt for adding independence Signed-off-by: bdattoma <[email protected]> * move tests to Edit suite Signed-off-by: bdattoma <[email protected]> * refactor clean project kw and move some tests Signed-off-by: bdattoma <[email protected]> * remove common workbench and refactor Signed-off-by: bdattoma <[email protected]> * fix ODS-1970 and 1971 - mark autobug on 1819 Signed-off-by: bdattoma <[email protected]> * fix misc failures Signed-off-by: bdattoma <[email protected]> * more fixes Signed-off-by: bdattoma <[email protected]> * fix 1813 and add wait for workbench deletion Signed-off-by: bdattoma <[email protected]> * add small sleep before checking secret deletion Signed-off-by: bdattoma <[email protected]> * add project creation in suite setup for Edit suite Signed-off-by: bdattoma <[email protected]> * minor changes suite setup/teardown in DSprojects suite Signed-off-by: bdattoma <[email protected]> * fix ods-1826 Signed-off-by: bdattoma <[email protected]> * fix setup edit suite Signed-off-by: bdattoma <[email protected]> * start test from ds project home page Signed-off-by: bdattoma <[email protected]> * fix dc edit test + edit dc name Signed-off-by: bdattoma <[email protected]> * fix edit workbench kw Signed-off-by: bdattoma <[email protected]> * add description to multiprojects test Signed-off-by: bdattoma <[email protected]> --------- Signed-off-by: bdattoma <[email protected]>

bdattoma added 14 commits October 11, 2023 12:51

remove ODS-1812

d90c3ee

Signed-off-by: bdattoma <[email protected]>

stop workbench from starting status

b316816

Signed-off-by: bdattoma <[email protected]>

add kw to delete single project from cli

dd0b8ab

Signed-off-by: bdattoma <[email protected]>

first attempt for adding independence

233f66f

Signed-off-by: bdattoma <[email protected]>

move tests to Edit suite

6a021f0

Signed-off-by: bdattoma <[email protected]>

refactor clean project kw and move some tests

a78aed4

Signed-off-by: bdattoma <[email protected]>

remove common workbench and refactor

91dbe63

Signed-off-by: bdattoma <[email protected]>

fix ODS-1970 and 1971 - mark autobug on 1819

61777d4

Signed-off-by: bdattoma <[email protected]>

fix misc failures

aebf7b5

Signed-off-by: bdattoma <[email protected]>

more fixes

9e9d412

Signed-off-by: bdattoma <[email protected]>

fix 1813 and add wait for workbench deletion

a4cdd31

Signed-off-by: bdattoma <[email protected]>

add small sleep before checking secret deletion

0507100

Signed-off-by: bdattoma <[email protected]>

add project creation in suite setup for Edit suite

4ed57ce

Signed-off-by: bdattoma <[email protected]>

minor changes suite setup/teardown in DSprojects suite

a2fe3a8

Signed-off-by: bdattoma <[email protected]>

bdattoma requested review from lugi0 and jstourac October 13, 2023 16:37

github-advanced-security bot found potential problems Oct 13, 2023

View reviewed changes

bdattoma requested review from jgarciao, manosnoam, diegolovison and tarukumar October 13, 2023 16:37

bdattoma self-assigned this Oct 13, 2023

bdattoma added needs testing Needs to be tested in Jenkins enhancements Bugfixes, enhancements, refactoring, ... in tests or libraries (PR will be listed in release-notes) do not merge Do not merge this yet please labels Oct 13, 2023

bdattoma added 2 commits October 13, 2023 18:49

fix ods-1826

63877c5

Signed-off-by: bdattoma <[email protected]>

fix setup edit suite

513ffe2

Signed-off-by: bdattoma <[email protected]>

start test from ds project home page

dd53c69

Signed-off-by: bdattoma <[email protected]>

fix dc edit test + edit dc name

3b1e293

Signed-off-by: bdattoma <[email protected]>

bdattoma removed the do not merge Do not merge this yet please label Oct 19, 2023

bdattoma and others added 2 commits October 19, 2023 11:32

Merge branch 'master' into f/dsg-setup

dcca8de

fix edit workbench kw

2411a8c

Signed-off-by: bdattoma <[email protected]>

bdattoma added verified This PR has been tested with Jenkins and removed needs testing Needs to be tested in Jenkins labels Oct 19, 2023

add description to multiprojects test

a3df2bb

Signed-off-by: bdattoma <[email protected]>

bdattoma added the ⚠️ high priority review I need this to be reviewed ASAP label Oct 19, 2023

FedeAlonso approved these changes Oct 20, 2023

View reviewed changes

Merge branch 'master' into f/dsg-setup

36372e9

jgarciao approved these changes Oct 23, 2023

View reviewed changes

bdattoma merged commit 98a4185 into red-hat-data-services:master Oct 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make DS Project tests independent #978

Make DS Project tests independent #978

bdattoma commented Oct 13, 2023 •

edited

Loading

github-advanced-security bot left a comment

github-actions bot commented Oct 13, 2023

bdattoma commented Oct 16, 2023 •

edited

Loading

FedeAlonso left a comment

sonarqubecloud bot commented Oct 20, 2023

jgarciao left a comment

bdattoma commented Oct 23, 2023

Make DS Project tests independent #978

Make DS Project tests independent #978

Conversation

bdattoma commented Oct 13, 2023 • edited Loading

github-advanced-security bot left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 13, 2023

Robot Results

bdattoma commented Oct 16, 2023 • edited Loading

FedeAlonso left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Oct 20, 2023

jgarciao left a comment

Choose a reason for hiding this comment

bdattoma commented Oct 23, 2023

bdattoma commented Oct 13, 2023 •

edited

Loading

bdattoma commented Oct 16, 2023 •

edited

Loading