Skip to content

Commit

Permalink
Automate Validate ServiceMeshControlPlaneAlreadyCreated Test Case (#1762
Browse files Browse the repository at this point in the history 
)

RHOAIENG-2517 Test Case to validate that only one Service Mesh Control
Plane per namespace/project can be created
  • Loading branch information
@asanzgom
asanzgom authored Sep 25, 2024
2 parents 5a5a234 + ac4fccf commit 9219cc1
Show file tree
Hide file tree
Showing 5 changed files with 224 additions and 38 deletions.
20 changes: 20 additions & 0 deletions ods_ci/tasks/Resources/Files/smcp_template.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
kind: ServiceMeshControlPlane
apiVersion: maistra.io/v2
metadata:
name: basic
namespace: istio-system
spec:
version: v2.6
policy:
type: Istiod
telemetry:
type: Istiod
addons:
prometheus:
enabled: false
kiali:
enabled: false
grafana:
enabled: false
tracing:
type: None
96 changes: 69 additions & 27 deletions ods_ci/tasks/Resources/RHODS_OLM/install/oc_install.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -281,34 +281,48 @@ Wait For DSCInitialization CustomResource To Be Ready

Apply DataScienceCluster CustomResource
[Documentation]
[Arguments] ${dsc_name}=${DSC_NAME}
[Arguments] ${dsc_name}=${DSC_NAME} ${custom}=False ${custom_cmp}=''
${file_path} = Set Variable tasks/Resources/Files/
Log to Console Requested Configuration:
FOR ${cmp} IN @{COMPONENT_LIST}
TRY
Log To Console ${cmp} - ${COMPONENTS.${cmp}}
EXCEPT
Log To Console ${cmp} - Removed
END
END
Log to Console message=Creating DataScience Cluster using yml template
Create DataScienceCluster CustomResource Using Test Variables
Apply Custom Manifest in DataScienceCluster CustomResource Using Test Variables
${yml} = Get File ${file_path}dsc_apply.yml
Log To Console Applying DSC yaml
Log To Console ${yml}
${return_code} ${output} = Run And Return Rc And Output oc apply -f ${file_path}dsc_apply.yml
Log To Console ${output}
Should Be Equal As Integers ${return_code} 0 msg=Error detected while applying DSC CR
Remove File ${file_path}dsc_apply.yml
FOR ${cmp} IN @{COMPONENT_LIST}
IF $cmp not in $COMPONENTS
Component Should Not Be Enabled ${cmp}
ELSE IF '${COMPONENTS.${cmp}}' == 'Managed'
Component Should Be Enabled ${cmp}
ELSE IF '${COMPONENTS.${cmp}}' == 'Removed'
Component Should Not Be Enabled ${cmp}
END
IF ${custom} == True
Log to Console message=Creating DataScience Cluster using custom configuration
Generate CustomManifest In DSC YAML
Rename DevFlags in DataScienceCluster CustomResource
${yml} = Get File ${file_path}dsc_apply.yml
Log To Console Applying DSC yaml
Log To Console ${yml}
${return_code} ${output} = Run And Return Rc And Output oc apply -f ${file_path}dsc_apply.yml
Log To Console ${output}
Should Be Equal As Integers ${return_code} 0 msg=Error detected while applying DSC CR
#Remove File ${file_path}dsc_apply.yml
Wait For DSC Conditions Reconciled ${OPERATOR_NS} ${DSC_NAME}
ELSE
Log to Console Requested Configuration:
FOR ${cmp} IN @{COMPONENT_LIST}
TRY
Log To Console ${cmp} - ${COMPONENTS.${cmp}}
EXCEPT
Log To Console ${cmp} - Removed
END
END
Log to Console message=Creating DataScience Cluster using yml template
Create DataScienceCluster CustomResource Using Test Variables
Apply Custom Manifest in DataScienceCluster CustomResource Using Test Variables
${yml} = Get File ${file_path}dsc_apply.yml
Log To Console Applying DSC yaml
Log To Console ${yml}
${return_code} ${output} = Run And Return Rc And Output oc apply -f ${file_path}dsc_apply.yml
Log To Console ${output}
Should Be Equal As Integers ${return_code} 0 msg=Error detected while applying DSC CR
Remove File ${file_path}dsc_apply.yml
FOR ${cmp} IN @{COMPONENT_LIST}
IF $cmp not in $COMPONENTS
Component Should Not Be Enabled ${cmp}
ELSE IF '${COMPONENTS.${cmp}}' == 'Managed'
Component Should Be Enabled ${cmp}
ELSE IF '${COMPONENTS.${cmp}}' == 'Removed'
Component Should Not Be Enabled ${cmp}
END
END
END

Create DataScienceCluster CustomResource Using Test Variables
Expand All @@ -325,7 +339,27 @@ Create DataScienceCluster CustomResource Using Test Variables
ELSE IF '${COMPONENTS.${cmp}}' == 'Removed'
Run sed -i'' -e 's/<${cmp}_value>/Removed/' ${file_path}dsc_apply.yml
END
# The model registry component needs to set the namespace used, so adding this special statement just for it
IF '${cmp}' == 'modelregistry'
Run sed -i'' -e 's/<${cmp}_namespace>/${MODEL_REGISTRY_NAMESPACE}/' ${file_path}dsc_apply.yml
END
END

Generate CustomManifest In DSC YAML
[Arguments] ${dsc_name}=${DSC_NAME}
Log To Console ${custom_cmp}.items
${file_path} = Set Variable tasks/Resources/Files/
Copy File source=${file_path}dsc_template.yml destination=${file_path}dsc_apply.yml
Run sed -i'' -e 's/<dsc_name>/${dsc_name}/' ${file_path}dsc_apply.yml
FOR ${cmp} IN @{COMPONENT_LIST}
${value}= Get From Dictionary ${custom_cmp} ${cmp}
${status}= Get From Dictionary ${value} managementState
Log To Console ${status}
IF '${status}' == 'Managed'
Run sed -i'' -e 's/<${cmp}_value>/Managed/' ${file_path}dsc_apply.yml
ELSE IF '${status}' == 'Removed'
Run sed -i'' -e 's/<${cmp}_value>/Removed/' ${file_path}dsc_apply.yml
END
# The model registry component needs to set the namespace used, so adding this special statement just for it
IF '${cmp}' == 'modelregistry'
Run sed -i'' -e 's/<${cmp}_namespace>/${MODEL_REGISTRY_NAMESPACE}/' ${file_path}dsc_apply.yml
Expand All @@ -346,6 +380,14 @@ Apply Custom Manifest in DataScienceCluster CustomResource Using Test Variables
END
END

Rename DevFlags in DataScienceCluster CustomResource
[Documentation] Filling devFlags fields for every component in DSC
Log To Console Filling devFlags fields for every component in DSC
${file_path} = Set Variable tasks/Resources/Files/
FOR ${cmp} IN @{COMPONENT_LIST}
Run sed -i'' -e "s|<${cmp}_devflags>||g" ${file_path}dsc_apply.yml
END

Wait For DataScienceCluster CustomResource To Be Ready
[Documentation] Wait for DataScienceCluster CustomResource To Be Ready
[Arguments] ${timeout}
Expand Down
2 changes: 1 addition & 1 deletion ods_ci/tests/Resources/ODS.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,7 @@ Delete RHODS Config Map

Wait Until Operator Ready
[Documentation] Wait for operator to be available/ready
[Arguments] ${operator_name} ${namespace} ${wait_time}=2m
[Arguments] ${operator_name} ${namespace} ${wait_time}=10m
${rc} ${output}= Run And Return Rc And Output
... oc wait --timeout=${wait_time} --for condition=available -n ${namespace} deploy/${operator_name}
Should Be Equal "${rc}" "0" msg=${output}
Expand Down
8 changes: 4 additions & 4 deletions ods_ci/tests/Resources/RHOSi.resource
View file
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,7 @@ Required Global Variables Should Exist
Assign Vars According To Product
[Documentation] Assign vars related to product
IF "${PRODUCT}" == "RHODS"
Set Suite Variable ${OPERATOR_APPNAME} Red Hat OpenShift AI
Set Suite Variable ${OPERATOR_NAME} Red Hat OpenShift AI
Set Suite Variable ${OPERATOR_APPNAME} "Red Hat OpenShift AI"
Set Suite Variable ${OPERATOR_DEPLOYMENT_NAME} rhods-operator
Set Suite Variable ${OPERATOR_POD_CONTAINER_NAME} rhods-operator
Set Suite Variable ${OPERATOR_LABEL_SELECTOR} name=rhods-operator
Expand All @@ -105,10 +104,10 @@ Assign Vars According To Product
Set Suite Variable ${DASHBOARD_DEPLOYMENT_NAME} rhods-dashboard
Set Suite Variable ${DASHBOARD_LABEL_SELECTOR} app.kubernetes.io/part-of=rhods-dashboard
Set Suite Variable ${APPLICATIONS_NAMESPACE} redhat-ods-applications
Set Suite Variable ${OPERATOR_SUBSCRIPTION_NAME} rhoai-operator-dev
Set Suite Variable ${MODEL_REGISTRY_NAMESPACE} rhoai-model-registries
ELSE IF "${PRODUCT}" == "ODH"
Set Suite Variable ${OPERATOR_APPNAME} Open Data Hub Operator
Set Suite Variable ${OPERATOR_NAME} Open Data Hub Operator
Set Suite Variable ${OPERATOR_APPNAME} "Open Data Hub Operator"
Set Suite Variable ${OPERATOR_DEPLOYMENT_NAME} opendatahub-operator-controller-manager
Set Suite Variable ${OPERATOR_POD_CONTAINER_NAME} manager
Set Suite Variable ${OPERATOR_LABEL_SELECTOR} control-plane=controller-manager
Expand All @@ -117,6 +116,7 @@ Assign Vars According To Product
Set Suite Variable ${DASHBOARD_DEPLOYMENT_NAME} odh-dashboard
Set Suite Variable ${DASHBOARD_LABEL_SELECTOR} app.kubernetes.io/part-of=dashboard
Set Suite Variable ${APPLICATIONS_NAMESPACE} opendatahub
Set Suite Variable ${OPERATOR_SUBSCRIPTION_NAME} rhoai-operator-dev
Set Suite Variable ${MODEL_REGISTRY_NAMESPACE} odh-model-registries
END

Expand Down
136 changes: 130 additions & 6 deletions ...0100__platform/0101__deploy/0104__operators/0104__rhods_operator/0110__service_mesh.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,17 @@ Suite Teardown Suite Teardown
*** Variables ***
${OPERATOR_NS} ${OPERATOR_NAMESPACE}
${DSCI_NAME} default-dsci
${DSC_NAME} default-dsc
${SERVICE_MESH_OPERATOR_NS} openshift-operators
${SERVICE_MESH_OPERATOR_DEPLOYMENT_NAME} istio-operator
${SERVICE_MESH_CR_NS} istio-system
${SERVICE_MESH_CR_NAME} data-science-smcp

${IS_PRESENT} 0
${IS_NOT_PRESENT} 1
${OLM_DIR} olm
${INSTALL_TYPE} CLi
${TEST_ENV} PSI
${IS_PRESENT} 0
${IS_NOT_PRESENT} 1
${MSSG_REGEX} denied the request: only one service mesh may be installed per project/namespace


*** Test Cases ***
Expand All @@ -32,7 +36,6 @@ Validate Service Mesh State Managed

[Teardown] Set Service Mesh State To Managed And Wait For CR Ready ${SERVICE_MESH_CR_NAME} ${SERVICE_MESH_CR_NS} ${OPERATOR_NS}


Validate Service Mesh State Unmanaged
[Documentation] The purpose of this Test Case is to validate Service Mesh state 'Unmanaged'.
... The operator will not recreate/update the Service Mesh CR if removed or changed.
Expand All @@ -58,15 +61,136 @@ Validate Service Mesh State Removed
[Teardown] Set Service Mesh State To Managed And Wait For CR Ready
... ${SERVICE_MESH_CR_NAME} ${SERVICE_MESH_CR_NS} ${OPERATOR_NS}

Validate Service Mesh Control Plane Already Created
[Documentation] This Test Case validates that only one ServiceMeshControlPlane is allowed to be installed per project/namespace
[Tags] Operator Tier3 RHOAIENG-2517
Fetch Image Url And Update Channel
Check Whether DSC Exists And Save Component Statuses
Fetch Cluster Type By Domain
IF "${CLUSTER_TYPE}" == "selfmanaged"
Uninstall RHODS In Self Managed Cluster
Create Smcp From Template
Install RHODS In Self Managed Cluster Using CLI ${CLUSTER_TYPE} ${IMAGE_URL}
ELSE IF "${CLUSTER_TYPE}" == "managed"
Uninstall RHODS In OSD
Create Smcp From Template
Install RHODS In Managed Cluster Using CLI ${CLUSTER_TYPE} ${IMAGE_URL}
END
Operator Deployment Should Be Ready
# Go check the Operator logs for the error message: denied the request: only one service mesh may be installed per project/namespace
Verify Pod Logs Contain Error
[Teardown] Teardown Service Mesh Control Plane Already Created


*** Keywords ***
Suite Setup
[Documentation] Suite Setup
RHOSi Setup
Wait Until Operator Ready ${SERVICE_MESH_OPERATOR_DEPLOYMENT_NAME} ${SERVICE_MESH_OPERATOR_NS}
Wait Until Operator Ready ${OPERATOR_DEPLOYMENT_NAME} ${OPERATOR_NS}
Wait For DSCI Ready State ${DSCI_NAME} ${OPERATOR_NS}
Wait Until Operator Ready ${OPERATOR_DEPLOYMENT_NAME} ${OPERATOR_NAMESPACE}
Wait For DSCI Ready State ${DSCI_NAME} ${OPERATOR_NAMESPACE}

Suite Teardown
[Documentation] Suite Teardown
RHOSi Teardown

Teardown Service Mesh Control Plane Already Created
# Cleanup the SMCP
Delete Smcp
# Cleanup Olminstall dir
Cleanup Olm Install Dir
IF ${DSC_EXISTS} == True
Apply DataScienceCluster CustomResource ${DSC_NAME} True ${custom_cmp}
END
Wait Until Keyword Succeeds 2 min 0 sec
... Is Resource Present ServiceMeshControlPlane ${SERVICE_MESH_CR_NAME} ${SERVICE_MESH_CR_NS} ${IS_PRESENT}

Check Whether DSC Exists And Save Component Statuses
${rc}= Run And Return Rc
... oc get datasciencecluster ${DSC_NAME}
IF ${rc} == 0
Set Global Variable ${DSC_EXISTS} True
&{custom_cmp} = Create Dictionary
${rc} ${out}= Run And Return Rc And Output
... oc get datasciencecluster ${DSC_NAME} -o jsonpath='{.spec.components}'
${custom_cmp}= Load Json String ${out}
Set Test Variable ${custom_cmp} ${custom_cmp}
ELSE
Set Global Variable ${DSC_EXISTS} False
END

Fetch Image Url And Update Channel
[Documentation] Fetch url for image and Update Channel
# Fetch subscription first
${rc} ${out}= Run And Return Rc And Output
... oc get subscription ${OPERATOR_SUBSCRIPTION_NAME} -n ${OPERATOR_NAMESPACE} -o jsonpath='{.spec.source}'
Should Be Equal As Integers ${rc} 0
Set Global Variable ${CS_NAME} ${out}
${rc} ${out}= Run And Return Rc And Output
... oc get subscription ${OPERATOR_SUBSCRIPTION_NAME} -n ${OPERATOR_NAMESPACE} -o jsonpath='{.spec.sourceNamespace}'
Should Be Equal As Integers ${rc} 0
Set Global Variable ${CS_NAMESPACE} ${out}
# Get CatalogSource
${rc} ${out}= Run And Return Rc And Output
... oc get catalogsource ${CS_NAME} --namespace ${CS_NAMESPACE} -o jsonpath='{.spec.image}'
Should Be Equal As Integers ${rc} 0
Set Global Variable ${IMAGE_URL} ${out}
${rc} ${out}= Run And Return Rc And Output
... oc get subscription ${OPERATOR_SUBSCRIPTION_NAME} --namespace ${OPERATOR_NS} -o jsonpath='{.spec.channel}'
Should Be Equal As Integers ${rc} 0
Set Global Variable ${UPDATE_CHANNEL} ${out}

Fetch Cluster Type By Domain
[Documentation] This Keyword outputs the kind of cluster depending on the console URL domain
${matches}= Get Regexp Matches ${OCP_CONSOLE_URL} rh-ods
${domain}= Get From List ${matches} 0
IF "${domain}" == "rh-ods"
Set Global Variable ${CLUSTER_TYPE} selfmanaged
ELSE
Set Global Variable ${CLUSTER_TYPE} managed
END

Create Smcp From Template
[Documentation] Create a default ServiceMeshControlPlane from a template
${file_path}= Set Variable ./tasks/Resources/Files/
${return_code}= Run And Return Rc oc apply -f ${file_path}smcp_template.yml
Should Be Equal As Integers ${return_code} 0

Delete Smcp
[Documentation] Delete the ServiceMeshControlPlane already created
Run and Watch Command
... oc delete ServiceMeshControlPlane basic --namespace ${SERVICE_MESH_CR_NS} --force --grace-period=0

Operator Deployment Should Be Ready
[Documentation] Loop until the Operator deployment is ready
${rc}= Set Variable 1
TRY
WHILE ${rc} != 0 limit=10m
Sleep 5s
${rc} ${output}= Run And Return Rc And Output
... oc wait --for condition=available -n ${OPERATOR_NAMESPACE} deploy/${OPERATOR_DEPLOYMENT_NAME}
END
EXCEPT WHILE loop was aborted type=start
Fail msg=Operator deployment did not get ready
END

Verify Pod Logs Contain Error
[Documentation] Checks whether there is a SMCP related error on the Pod Logs
${pod_name}= Get Pod Name ${OPERATOR_NAMESPACE} ${OPERATOR_LABEL_SELECTOR}
${pod_logs}= Oc Get Pod Logs
... name=${pod_name}
... namespace=${OPERATOR_NAMESPACE}
... container=${OPERATOR_POD_CONTAINER_NAME}
${match_list}= Get Regexp Matches ${pod_logs} ${MSSG_REGEX}
${entry_msg}= Remove Duplicates ${match_list}
${length}= Get Length ${entry_msg}
IF ${length} == ${0}
FAIL Pod ${pod_name} logs should contain message ${MSSG_REGEX}.
ELSE
Log message=Pod ${pod_name} logs contain error message '${MSSG_REGEX}' level=INFO
END

Cleanup Olm Install Dir
[Documentation] Clean up olm install repo
${return_code}= Run and Watch Command rm -vRf ${EXECDIR}/${OLM_DIR} timeout=10 min
Should Be Equal As Integers ${return_code} 0 msg=Error while cleaning up olminstall directory

0 comments on commit 9219cc1

Please sign in to comment.