Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parity between Caikit+TGIS and TGIS standalone test suites #1218

Merged
merged 16 commits into from
Feb 21, 2024
Merged

Add parity between Caikit+TGIS and TGIS standalone test suites #1218

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
4e61a51
add TGIS suite with initial adjustments
bdattoma Feb 19, 2024
18ea71f
remove from sanity
bdattoma Feb 19, 2024
838219e
fix ODS-2378
bdattoma Feb 19, 2024
46552b1
fix ODS-2373
bdattoma Feb 19, 2024
1b87609
fix file name
bdattoma Feb 20, 2024
7aa0889
fix compile deploy and query keyword
bdattoma Feb 20, 2024
7eff949
add clone proto files
bdattoma Feb 20, 2024
c1e879a
fix ODS-2377
bdattoma Feb 20, 2024
473df0c
fix ODS-2371
bdattoma Feb 20, 2024
a4d908e
change project name
bdattoma Feb 20, 2024
51567b1
fix ODS-2380
bdattoma Feb 20, 2024
1477b97
fix ODS-2372
bdattoma Feb 20, 2024
d57e21f
remove auto bug label from ODS-2372
bdattoma Feb 20, 2024
7778cf4
remove wrong arg from canary kw
bdattoma Feb 21, 2024
6123d9d
change metrics check for tgis ODS-2401
bdattoma Feb 21, 2024
066c8ba
fix typo
bdattoma Feb 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 21 additions & 15 deletions ods_ci/tests/Resources/CLI/ModelServing/llm.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,12 +273,12 @@
Run Keyword And Continue On Failure Status Should Be 200
END
Log ${res}
${response_container_field}= Set Variable ${runtime_details}[response_fields_map][response]
IF "${response_container_field}" != "${EMPTY}"
# runtimes may support multiple queries per time. Here forcing to use only 1 for sake of simplicity.
${res}= Set Variable ${res}[${response_container_field}][0]
END
IF ${validate_response} == ${TRUE}
${response_container_field}= Set Variable ${runtime_details}[response_fields_map][response]
IF "${response_container_field}" != "${EMPTY}"
# runtimes may support multiple queries per time. Here forcing to use only 1 for sake of simplicity.
${res}= Set Variable ${res}[${response_container_field}][0]

Check warning

Code scanning / Robocop

Trailing whitespace at the end of line Warning test

Trailing whitespace at the end of line
END
Run Keyword And Continue On Failure
... Model Response Should Match The Expectation model_response=${res} model_name=${model_name}
... runtime_details=${runtime_details} runtime=${runtime}
Expand All @@ -290,13 +290,14 @@
[Documentation] Group together the test steps for preparing, deploying
... and querying a model
[Arguments] ${model_storage_uri} ${model_name} ${isvc_name}=${model_name}
... ${runtime}=caikit-tgis-runtime ${protocol}=grpc ${inference_type}=all-tokens
... ${runtime}=caikit-tgis-runtime ${model_format}=caikit ${protocol}=grpc ${inference_type}=all-tokens

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (127/120)

Check notice

Code scanning / Robocop

There is too many arguments per continuation line ({{ arguments_count }} / {{ max_arguments_count }}) Note test

There is too many arguments per continuation line (4 / 1)
... ${canaryTrafficPercent}=${EMPTY} ${namespace}=${TEST_NS} ${sa_name}=${DEFAULT_BUCKET_SA_NAME}
... ${n_queries}=${1} ${query_idx}=${0} ${validate_response}=${TRUE}
Compile Inference Service YAML isvc_name=${isvc_name}
... sa_name=${sa_name}
... sa_name=${sa_name} serving_runtime=${runtime}
... model_storage_uri=${model_storage_uri}
... canaryTrafficPercent=${canaryTrafficPercent}
... model_format=${model_format}
Deploy Model Via CLI isvc_filepath=${INFERENCESERVICE_FILLED_FILEPATH}
... namespace=${namespace}
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${isvc_name}
Expand All @@ -306,24 +307,28 @@
... validate_response=${validate_response} protocol=${protocol}
... runtime=${runtime} inference_type=${inference_type}

Upgrade Caikit Runtime Image
Upgrade Runtime Image
[Documentation] Replaces the image URL of the Caikit Runtim with the given
... ${new_image_url}
[Arguments] ${new_image_url} ${namespace}
[Arguments] ${new_image_url} ${namespace} ${container} ${runtime}
${rc} ${container_idx}= Run And Return Rc And Output

Check warning

Code scanning / Robocop

Local variable '{{ name }}' is overwritten before usage Warning test

Local variable '${rc}' is overwritten before usage
... oc get ServingRuntime/${runtime} -n ${namespace} -o json | jq '.spec.containers | map(.name == "${container}") | index(true)' # robocop: disable
${rc} ${out}= Run And Return Rc And Output
... oc patch ServingRuntime caikit-tgis-runtime -n ${namespace} --type=json -p="[{'op': 'replace', 'path': '/spec/containers/0/image', 'value': '${new_image_url}'}]" # robocop: disable
... oc patch ServingRuntime ${runtime} -n ${namespace} --type=json -p="[{'op': 'replace', 'path': '/spec/containers/${container_idx}/image', 'value': '${new_image_url}'}]" # robocop: disable
Should Be Equal As Integers ${rc} ${0}

Get Model Pods Creation Date And Image URL
[Documentation] Fetches the creation date and the caikit runtime image URL.
... Useful in upgrade scenarios
[Arguments] ${model_name} ${namespace}
[Arguments] ${model_name} ${namespace} ${container}
${created_at}= Oc Get kind=Pod label_selector=serving.kserve.io/inferenceservice=${model_name}
... namespace=${namespace} fields=["metadata.creationTimestamp"]
${rc} ${caikitsha}= Run And Return Rc And Output
... oc get pod --selector serving.kserve.io/inferenceservice=${model_name} -n ${namespace} -ojson | jq '.items[].spec.containers[].image' | grep caikit-tgis # robocop: disable
${rc} ${container_idx}= Run And Return Rc And Output

Check warning

Code scanning / Robocop

Local variable '{{ name }}' is overwritten before usage Warning test

Local variable '${rc}' is overwritten before usage
... oc get pod --selector serving.kserve.io/inferenceservice=${model_name} -n ${namespace} -o json | jq '.items[].spec.containers | map(.name == "${container}") | index(true)' # robocop: disable
${rc} ${image_url}= Run And Return Rc And Output
... oc get pod --selector serving.kserve.io/inferenceservice=${model_name} -n ${namespace} -ojson | jq '.items[].spec.containers[${container_idx}].image' # robocop: disable
Should Be Equal As Integers ${rc} ${0}
RETURN ${created_at} ${caikitsha}
RETURN ${created_at} ${image_url}

User Can Fetch Number Of Requests Over Defined Time
[Documentation] Fetches the `tgi_request_count` metric and checks that it reports the expected
Expand Down Expand Up @@ -452,14 +457,15 @@
... matches the expected probability ${exp_percentage}.
... It applies an arbitrary toleration margin of ${toleration}
[Arguments] ${exp_percentage} ${isvc_name} ${model_name} ${namespace}
... ${runtime}
${total}= Set Variable ${20}
${hits}= Set Variable ${0}
${toleration}= Set Variable ${20}
FOR ${counter} IN RANGE ${0} ${total}
Log ${counter}
${status}= Run Keyword And Return Status
... Query Model Multiple Times isvc_name=${isvc_name} model_name=${model_name} n_times=1
... namespace=${namespace}
... namespace=${namespace} runtime=${runtime}
IF ${status} == ${TRUE}
${hits}= Evaluate ${hits}+1
END
Expand Down
10 changes: 6 additions & 4 deletions ods_ci/tests/Tests/400__ods_dashboard/420__model_serving/LLMs/422__model_serving_llm.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ Verify Model Upgrade Using Canaray Rollout
... n_queries=${0}
Traffic Should Be Redirected Based On Canary Percentage exp_percentage=${canary_percentage}
... isvc_name=${isvc_name} model_name=${model_name} namespace=${test_namespace}
... runtime=caikit-tgi-runtime
bdattoma marked this conversation as resolved.
Show resolved Hide resolved
Log To Console Remove Canary Tarffic For Model Upgrade
Compile Deploy And Query LLM model isvc_name=${isvc_name}
... sa_name=${DEFAULT_BUCKET_SA_NAME}
Expand All @@ -178,6 +179,7 @@ Verify Model Upgrade Using Canaray Rollout
... namespace=${test_namespace}
Traffic Should Be Redirected Based On Canary Percentage exp_percentage=${100}
... isvc_name=${isvc_name} model_name=${model_name} namespace=${test_namespace}
... runtime=caikit-tgi-runtime
[Teardown] Clean Up Test Project test_ns=${test_namespace}
... isvc_names=${isvcs_names} wait_prj_deletion=${FALSE}

Expand Down Expand Up @@ -465,14 +467,14 @@ Verify Runtime Upgrade Does Not Affect Deployed Models
... inference_type=all-tokens n_times=1
... namespace=${test_namespace}
${created_at} ${caikitsha}= Get Model Pods Creation Date And Image URL model_name=${flan_model_name}
... namespace=${test_namespace}
Upgrade Caikit Runtime Image new_image_url=quay.io/opendatahub/caikit-tgis-serving:stable
... namespace=${test_namespace}
... namespace=${test_namespace} container=transformer-container
Upgrade Runtime Image new_image_url=quay.io/opendatahub/caikit-tgis-serving:stable
... namespace=${test_namespace} container=transformer-container runtime=caikit-tgis-runtime
Sleep 5s reason=Sleep, in case the runtime upgrade takes some time to start performing actions on the pods...
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} exp_replicas=1
${created_at_after} ${caikitsha_after}= Get Model Pods Creation Date And Image URL model_name=${flan_model_name}
... namespace=${test_namespace}
... namespace=${test_namespace} container=transformer-container
Should Be Equal ${created_at} ${created_at_after}
Should Be Equal As Strings ${caikitsha} ${caikitsha_after}
[Teardown] Clean Up Test Project test_ns=${test_namespace}
Expand Down
Loading
Loading