Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Upgrade test scenarios for Distributed workloads #1491

Merged
merged 1 commit into from
Jun 25, 2024
Merged

Add Upgrade test scenarios for Distributed workloads #1491

merged 1 commit into from
Jun 25, 2024

Conversation

Srihari1192
Copy link
Contributor

@Srihari1192 Srihari1192 commented May 31, 2024 •
edited
Loading

closes https://issues.redhat.com/browse/RHOAIENG-7032

Added Pre upgrade and Post upgrade test scenarios for Distributed workloads

  • For Pre upgrade scenario running TestMNISTRayClusterUp test for Ray cluster up and monitor Distributed workload metrics check
  • For Post upgrade scenario running TestMnistJobSubmit test to verify created Ray Cluster exists, submit Ray Job and monitor Distributed workload metrics

github-advanced-security[bot]
github-advanced-security bot found potential problems May 31, 2024
View reviewed changes
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robocop found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 31, 2024 •
edited
Loading

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass %
488 0 0 488 100

@Srihari1192 Srihari1192 added the needs testing Needs to be tested in Jenkins label May 31, 2024
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 12, 2024
View reviewed changes
ods_ci/tests/Resources/Page/DistributedWorkloads/DistributedWorkloads.resource Dismissed Show dismissed Hide dismissed
ods_ci/tests/Resources/Page/DistributedWorkloads/DistributedWorkloads.resource Dismissed Show dismissed Hide dismissed
ods_ci/tests/Resources/Page/DistributedWorkloads/DistributedWorkloads.resource Dismissed Show dismissed Hide dismissed
ods_ci/tests/Resources/Page/DistributedWorkloads/DistributedWorkloads.resource Dismissed Show dismissed Hide dismissed
ods_ci/tests/Resources/Page/DistributedWorkloads/DistributedWorkloads.resource Dismissed Show dismissed Hide dismissed
ods_ci/tests/Resources/Page/DistributedWorkloads/DistributedWorkloads.resource
FAIL Unable to cleanup directory ${CODEFLARE-SDK_DIR}
END

IF ${project_created} == True Run Keywords

Check notice

Code scanning / Robocop

'{{ block_name }}' condition can be simplified Note test

'INLINE IF' condition can be simplified
ods_ci/tests/Tests/100__deploy/120__upgrades/120__pre_upgrades.robot
... ${ODH_DASHBOARD_URL} ${BROWSER.NAME} ${BROWSER.OPTIONS}
Open Distributed Workload Metrics Home Page
Select Distributed Workload Project By Name ${PRJ_UPGRADE}
Set Global Variable ${DW_PROJECT_CREATED} True

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Global Variable can be replaced with VAR
ods_ci/tests/Tests/100__deploy/120__upgrades/120__pre_upgrades.robot Fixed Show fixed Hide fixed
ods_ci/tests/Tests/100__deploy/120__upgrades/122__post_ugrade.robot Fixed Show fixed Hide fixed
ods_ci/tests/Tests/100__deploy/120__upgrades/122__post_ugrade.robot
Wait Until Element Is Visible xpath=//*[text()="Running"] timeout=30

${cpu_requested} = Get CPU Requested ${PRJ_UPGRADE} ${LOCAL_QUEUE}
${memory_requested} = Get Memory Requested ${PRJ_UPGRADE} ${LOCAL_QUEUE} Upgrade

Check warning

Code scanning / Robocop

The assignment sign is not consistent within the file. Expected '{{ expected_sign }}' but got '{{ actual_sign }}' instead Warning test

The assignment sign is not consistent within the file. Expected '' but got ' =' instead
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 12, 2024
View reviewed changes
ods_ci/tests/Tests/100__deploy/120__upgrades/120__pre_upgrades.robot
@@ -119,6 +123,39 @@
Page Should Contain Element //tr[@id='caikit-runtime']
[Teardown] Dashboard Test Teardown

Verify Distributed Workload Metrics Resources By Creating Ray Cluster Workload

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' is too long ({{ test_length }}/{{ allowed_length }}) Warning test

Test case 'Verify Distributed Workload Metrics Resources By Creating Ray Cluster Workload' is too long (33/20)
ods_ci/tests/Tests/100__deploy/120__upgrades/120__pre_upgrades.robot
@@ -119,6 +123,39 @@
Page Should Contain Element //tr[@id='caikit-runtime']
[Teardown] Dashboard Test Teardown

Verify Distributed Workload Metrics Resources By Creating Ray Cluster Workload

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Verify Distributed Workload Metrics Resources By Creating Ray Cluster Workload' has too many keywords inside (20/10)
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 13, 2024
View reviewed changes
ods_ci/tests/Resources/Page/DistributedWorkloads/DistributedWorkloads.resource
FAIL Unable to setup Python virtual environment
END

${result} = Run Process source ${VIRTUAL_ENV_NAME}/bin/activate && cd ${CODEFLARE-SDK_DIR} && poetry env use 3.9 && poetry install --with test,docs && poetry run pytest -v -s ./tests/upgrade/raycluster_sdk_upgrade_test.py::${TEST_NAME} --timeout\=300 && deactivate

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (272/120)
@Srihari1192 Srihari1192 requested a review from sutaakar June 13, 2024 06:58
@Srihari1192 Srihari1192 force-pushed the RHOAIENG-7032 branch 2 times, most recently from 3327d73 to 6ae3bcd Compare June 14, 2024 05:57
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 14, 2024
View reviewed changes
ods_ci/tests/Tests/100__deploy/120__upgrades/122__post_ugrade.robot
${LOCAL_QUEUE} Set Variable local-queue-mnist
${JOB_NAME} Set Variable mnist
Run Codeflare Upgrade Tests TestMnistJobSubmit
Set Global Variable ${DW_PROJECT_CREATED} True

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Global Variable can be replaced with VAR
@Srihari1192 Srihari1192 removed the needs testing Needs to be tested in Jenkins label Jun 14, 2024
@Srihari1192 Srihari1192 marked this pull request as ready for review June 14, 2024 09:35
@Srihari1192
Copy link
Contributor Author

Srihari1192 commented Jun 14, 2024 •
edited
Loading

Run the tests in rhods-ci-pr-test/2974/ with customized branch by adding tags test looks good..
When running independently pre upgrade and post upgrade scenarios noticed some import error in Post-upgrade existing tests

11:07:48  Multiple keywords with name 'Open Browser' found. Give the full name of the keyword you want to use:
11:07:48      JupyterLibrary.Open Browser
11:07:48      SeleniumLibrary.Open Browser


[ ERROR ] Error in file 'ods-ci/ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHModelServing.resource' on line 8: Importing library 'ods-ci/ods_ci/libs/Helpers.py' failed: ModuleNotFoundError: No module named 'fuzzywuzzy'
Traceback (most recent call last):
  File "ods-ci/ods_ci/libs/Helpers.py", line 9, in <module>
    from fuzzywuzzy import fuzz

@Srihari1192 Srihari1192 requested a review from ChughShilpa June 19, 2024 06:01
@sutaakar sutaakar merged commit ba3b74e into red-hat-data-services:master Jun 25, 2024
7 checks passed
@Srihari1192 Srihari1192 deleted the RHOAIENG-7032 branch October 21, 2024 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sutaakar sutaakar sutaakar approved these changes

@jiripetrlik jiripetrlik jiripetrlik approved these changes

@ChughShilpa ChughShilpa ChughShilpa approved these changes

@tarukumar tarukumar Awaiting requested review from tarukumar

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Srihari1192 @sutaakar @jiripetrlik @ChughShilpa