Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Data Science Pipelines to slim down sanity (Sanity on a diet) #1481

Merged
merged 10 commits into from
May 29, 2024
Merged

Refactor Data Science Pipelines to slim down sanity (Sanity on a diet) #1481

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
a054011
Split some tests. Move others from Sanity to Tier1
jgarciao May 21, 2024
b7e3a2c
Add Sealights integration dependency. Temporally delete rpaframework dep
jgarciao May 23, 2024
d558a41
Revert "Add Sealights integration dependency. Temporally delete rpafr…
jgarciao May 23, 2024
a0d0ae8
Add missing Polarion IDs. Fix Elyra test for HabanaAI
jgarciao May 27, 2024
3fcbd4f
Adjust Elra test timeouts
jgarciao May 28, 2024
eeaf20b
Fix data science pipelines xpaths chaged in RHOAI 2.10
jgarciao May 28, 2024
0246c43
Merge branch 'master' into sanity-diet-pipelines
jgarciao May 28, 2024
4b140ec
Increase Elyra test timeouts
jgarciao May 28, 2024
6501b5e
Merge branch 'master' into sanity-diet-pipelines
FedeAlonso May 29, 2024
1e1b9b2
Move SDS Elyra test to Sanity and the rest to Tier1
jgarciao May 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Pipelines.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Wait Until Pipeline Is Not Listed
[Documentation] Waits until pipeline is no longer listed in the pipelines table
[Arguments] ${pipeline_name} ${timeout}=60s
Maybe Wait For Dashboard Loading Spinner Page
${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title-${pipeline_name}"])[1]/../..
${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title" and text()="${pipeline_name}"])[1]/../..
Wait Until Page Does Not Contain Element ${pipeline_row_xp} timeout=${timeout}

Import Pipeline
Expand Down Expand Up @@ -106,7 +106,7 @@ Expand Pipeline Details
[Documentation] Expands a pipeline row in the dashboard UI
[Arguments] ${pipeline_name}

${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title-${pipeline_name}"])[1]/../..
${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title" and text()="${pipeline_name}"])[1]/../..
${pipeline_details_xp}= Set Variable ${pipeline_row_xp}/td/button[@aria-label="Details"]
${expanded_pipeline_details_xp}= Set Variable ${pipeline_row_xp}/td/button[@aria-expanded="true"]

Expand All @@ -117,7 +117,7 @@ Expand Pipeline Details
Collapse Pipeline Details
[Documentation] Collapses a pipeline row in the dashboard UI
[Arguments] ${pipeline_name}
${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title-${pipeline_name}"])[1]/../..
${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title" and text()="${pipeline_name}"])[1]/../..
${pipeline_details_xp}= Set Variable ${pipeline_row_xp}/td/button[@aria-label="Details"]
${expanded_pipeline_details_xp}= Set Variable ${pipeline_row_xp}/td/button[@aria-expanded="true"]

Expand All @@ -129,7 +129,7 @@ Click Action From Pipeline Actions Menu
[Documentation] Click an action from Pipeline Actions menu (3-dots menu on the right)
[Arguments] ${pipeline_name} ${action}
# robocop: off=line-too-long
${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title-${pipeline_name}"])[1]/../..
${pipeline_row_xp}= Set Variable (//*[@data-testid="table-row-title" and text()="${pipeline_name}"])[1]/../..
${kebab_xp}= Set Variable ${pipeline_row_xp}/td/button[@aria-label="Kebab toggle"]
${kebap_action_xp}= Set Variable ${pipeline_row_xp}/td/div/div/ul/li/button[.//*[contains(text(), '${action}')]]
Wait Until Page Contains Element ${kebab_xp}
Expand All @@ -141,9 +141,9 @@ Click Action From Pipeline Version Actions Menu
[Documentation] Click an action from Pipeline Version Actions menu (3-dots menu on the right)
[Arguments] ${pipeline_version_name_contains} ${action}
# robocop: off=line-too-long
${pipeline_row_xp}= Set Variable (//*[contains(@data-testid,"table-row-title-${pipeline_version_name_contains}")])[2]/../..
${kebab_xp}= Set Variable ${pipeline_row_xp}/td/button[@aria-label="Kebab toggle"]
${kebap_action_xp}= Set Variable ${pipeline_row_xp}/td/div/div/ul/li/button[.//*[contains(text(), '${action}')]]
${version_row_xp}= Set Variable (//*[@data-testid="table-row-title"]/a[contains(text(),"${pipeline_version_name_contains}")])/../../..
${kebab_xp}= Set Variable ${version_row_xp}/td/button[@aria-label="Kebab toggle"]
${kebap_action_xp}= Set Variable ${version_row_xp}/td/div/div/ul/li/button[.//*[contains(text(), '${action}')]]
Wait Until Page Contains Element ${kebab_xp}
Click Element ${kebab_xp}
Wait Until Page Contains Element ${kebap_action_xp}
Expand All @@ -153,7 +153,7 @@ Click Action From Pipeline Run Actions Menu
[Documentation] In Data Science Pipelines > Runs, click an action from Actions menu (3-dots menu on the right)
[Arguments] ${pipeline_run_name} ${action}

${pipeline_run_row_xp}= Set Variable //table[@data-testid="pipeline-run-table"]/tbody/tr/td/div/div/p[text()="${pipeline_run_name}"]/../../../..
${pipeline_run_row_xp}= Set Variable //span[contains(text(),"${pipeline_run_name}")]/../../../..
${kebab_xp}= Set Variable ${pipeline_run_row_xp}/td/button[@aria-label="Kebab toggle"]
${kebap_action_xp}= Set Variable ${pipeline_run_row_xp}/td/div/div/ul/li/button[.//*[contains(text(), '${action}')]]
Wait Until Page Contains Element ${kebab_xp}
Expand Down Expand Up @@ -193,7 +193,7 @@ Pipeline Should Be Listed
[Arguments] ${pipeline_name} ${pipeline_description}=${EMPTY}

Projects.Move To Tab Pipelines
${pipeline_title_xp}= Set Variable (//*[@data-testid="table-row-title-${pipeline_name}"])[1]
${pipeline_title_xp}= Set Variable (//*[@data-testid="table-row-title" and text()="${pipeline_name}"])[1]
Run Keyword And Continue On Failure Wait Until Page Contains Element ${pipeline_title_xp}

IF "${pipeline_description}" != "${EMPTY}"
Expand All @@ -206,7 +206,7 @@ Pipeline Should Not Be Listed
[Documentation] Verifies a pipeline not listed in Data Science Project > Pipelines
[Arguments] ${pipeline_name}
Projects.Move To Tab Pipelines
${pipeline_title_xp}= Set Variable (//*[@data-testid="table-row-title-${pipeline_name}"])[1]
${pipeline_title_xp}= Set Variable (//*[@data-testid="table-row-title" and text()="${pipeline_name}"])[1]
Run Keyword And Continue On Failure Wait Until Page Does Not Contain Element ${pipeline_title_xp}

Pipeline Last Run Should Be
Expand Down
11 changes: 8 additions & 3 deletions ...ests/400__ods_dashboard/430__data_science_pipelines/431__data-science-pipelines-api.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,21 @@ ${URL_TEST_PIPELINE_RUN_YAML}= https://raw.githubusercontent.com


*** Test Cases ***
Verify Ods Users Can Create And Run a Data Science Pipeline Using The Api
[Documentation] Creates, runs pipelines with admin and regular user. Double check the pipeline result and clean
Verify Admin Users Can Create And Run a Data Science Pipeline Using The Api
[Documentation] Creates, runs pipelines with admin user. Double check the pipeline result and clean
... the pipeline resources.
[Tags] Sanity Tier1 ODS-2083
End To End Pipeline Workflow Via Api ${OCP_ADMIN_USER.USERNAME} ${OCP_ADMIN_USER.PASSWORD} pipelinesapi1

Verify Regular Users Can Create And Run a Data Science Pipeline Using The Api
[Documentation] Creates, runs pipelines with regular user. Double check the pipeline result and clean
... the pipeline resources.
[Tags] Tier1 ODS-2677
End To End Pipeline Workflow Via Api ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} pipelinesapi2

Verify Ods Users Can Do Http Request That Must Be Redirected to Https
[Documentation] Verify Ods Users Can Do Http Request That Must Be Redirected to Https
[Tags] Sanity Tier1 ODS-2234
[Tags] Tier1 ODS-2234
New Project project-redirect-http
Install DataSciencePipelinesApplication CR project-redirect-http
${status} Login And Wait Dsp Route ${OCP_ADMIN_USER.USERNAME} ${OCP_ADMIN_USER.PASSWORD}
Expand Down
10 changes: 8 additions & 2 deletions ...ests/400__ods_dashboard/430__data_science_pipelines/432__data-science-pipelines-kfp.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@
... method_name=flipcoin_pipeline
... status_check_timeout=440
... pipeline_params=${emtpy_dict}
[Teardown] Remove Pipeline Project ${PROJECT_NAME}

Verify Upload Download In Data Science Pipelines Using The kfp Python Package
[Documentation] Creates, runs pipelines with regular user. Double check the pipeline result and clean
... the pipeline resources.
[Tags] Tier1 ODS-2683
${upload_download_dict}= Create Dictionary mlpipeline_minio_artifact_secret=value bucket_name=value
End To End Pipeline Workflow Using Kfp
... username=${TEST_USER.USERNAME}
Expand All @@ -45,11 +51,11 @@
[Teardown] Remove Pipeline Project ${PROJECT_NAME}



Verify Ods Users Can Create And Run A Data Science Pipeline With Ray Using The kfp Python Package

Check warning

Code scanning / Robocop

Invalid number of empty lines between test cases ({{ empty_lines }}/{{ allowed_empty_lines }}) Warning test

Invalid number of empty lines between test cases (3/1)
[Documentation] Creates, runs pipelines with regular user. Double check the pipeline result and clean
... the pipeline resources.
[Tags] Sanity
... Tier1
[Tags] Tier1
... ODS-2541
Skip If Component Is Not Enabled ray
Skip If Component Is Not Enabled codeflare
Expand Down
3 changes: 1 addition & 2 deletions .../400__ods_dashboard/430__data_science_pipelines/433__data-science-pipelines-general.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ Verify Ods User Can Bind The Route Role
... Test with user3 can access the dsp route for user4, it should fail because it doesn't have the permission # robocop: disable:line-too-long
... Add the permission using a role binding
... Test with user3 can access the dsp route for user4, it should work because it has the permission
[Tags] Sanity
... Tier1
[Tags] Tier1
... ODS-2209
Create A Pipeline Server And Wait For Dsp Route ${TEST_USER_3.USERNAME} ${TEST_USER_3.PASSWORD}
... ${TEST_USER_3.AUTH_TYPE} ${PROJECT_USER3}
Expand Down
33 changes: 24 additions & 9 deletions ...ts/400__ods_dashboard/430__data_science_pipelines/435__data-science-pipelines-elyra.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,21 +40,36 @@ Verify Pipelines Integration With Elyra When Using Standard Data Science Image
[Documentation] Verifies that a workbench using the Standard Data Science Image can be used to
... create and run a Data Science Pipeline
[Tags] Smoke Tier1
... ODS-2197 ODS-2199
Verify Pipelines Integration With Elyra Running Hello World Pipeline Test Standard Data Science
... ODS-2197
[Timeout] 8m
Verify Pipelines Integration With Elyra Running Hello World Pipeline Test
... img=Standard Data Science
... runtime_image=Datascience with Python 3.9 (UBI9)

Verify Pipelines Integration With Elyra When Using PyTorch Image
[Documentation] Verifies that a workbench using the PyTorch Image can be used to
... create and run a Data Science Pipeline
... Note: this a templated test case
... (more info at https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#test-templates)
[Template] Verify Pipelines Integration With Elyra Running Hello World Pipeline Test
[Tags] Sanity Tier1
... ODS-2199
[Timeout] 8m
PyTorch Datascience with Python 3.9 (UBI9)


Verify Pipelines Integration With Elyra When Using Standard Data Science Based Images
[Documentation] Verifies that a workbench using an image based on the Standard Data Science Image
... can be used to create and run a Data Science Pipeline
... Note: this a templated test case
... (more info at https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#test-templates)
[Template] Verify Pipelines Integration With Elyra Running Hello World Pipeline Test
[Tags] Sanity Tier1
[Tags] Tier1
... ODS-2271
PyTorch
TensorFlow
TrustyAI
HabanaAI
[Timeout] 20m
TensorFlow Datascience with Python 3.9 (UBI9)
TrustyAI Datascience with Python 3.9 (UBI9)
HabanaAI Datascience with Python 3.8 (UBI8)


*** Keywords ***
Expand Down Expand Up @@ -85,7 +100,7 @@ Elyra Pipelines Suite Teardown
Verify Pipelines Integration With Elyra Running Hello World Pipeline Test # robocop: off=too-many-calls-in-keyword
[Documentation] Creates and starts a workbench using ${img} and verifies that the Hello World sample pipeline
... runs successfully
[Arguments] ${img}
[Arguments] ${img} ${runtime_image}
Create Workbench workbench_title=elyra_${img} workbench_description=Elyra test
... prj_title=${PRJ_TITLE} image_name=${img} deployment_size=Small
... storage=Persistent pv_existent=${FALSE}
Expand All @@ -96,7 +111,7 @@ Verify Pipelines Integration With Elyra Running Hello World Pipeline Test #
Clone Git Repository And Open https://github.com/redhat-rhods-qe/ods-ci-notebooks-main
... ods-ci-notebooks-main/notebooks/500__jupyterhub/pipelines/v2/elyra/run-pipelines-on-data-science-pipelines/hello-generic-world.pipeline # robocop: disable
Verify Hello World Pipeline Elements
Set Runtime Image In All Nodes runtime_image=Datascience with Python 3.9 (UBI9)
Set Runtime Image In All Nodes runtime_image=${runtime_image}
Run Pipeline pipeline_name=${img} Pipeline
Wait Until Page Contains Element xpath=//a[.="Run Details."] timeout=30s
${pipeline_run_name} = Get Pipeline Run Name
Expand Down
1 change: 0 additions & 1 deletion ...430__data_science_pipelines/436__test-run-data-science-pipelines-operator-e2e-tests.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ ${KUBECONFIGPATH} %{HOME}/.kube/config
Run Data Science Pipelines Operator Integration Tests
[Documentation] Run Data Science Pipelines Operator Integration Tests
[Tags]
... Sanity
... DataSciencePipelines
... Tier1
... ODS-2632
Expand Down
Loading