Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added onnx model gRPC protocol test on triton #1790

Merged
merged 18 commits into from
Sep 19, 2024
Merged

Added onnx model gRPC protocol test on triton #1790

merged 18 commits into from
Sep 19, 2024

Conversation

Raghul-M
Copy link
Contributor

@Raghul-M Raghul-M commented Sep 9, 2024

Added Automation Test for RHOAIENG-11565

@Raghul-M Raghul-M added needs testing Needs to be tested in Jenkins do not merge Do not merge this yet please labels Sep 9, 2024
@Raghul-M Raghul-M requested review from lugi0 and tarukumar September 9, 2024 06:46
@Raghul-M Raghul-M self-assigned this Sep 9, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 9, 2024 •
edited
Loading

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass %
541 0 0 541 100

github-advanced-security[bot]
github-advanced-security bot found potential problems Sep 17, 2024
View reviewed changes
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
${INFERENCE_REST_INPUT_PYTORCH}= @tests/Resources/Files/triton/kserve-triton-resnet-rest-input.json
${PYTORCH_MODEL_NAME}= resnet50
${PYTORCH_RUNTIME_NAME}= triton-kserve-rest
${PYTORCH_RUNTIME_FILEPATH}= ${RESOURCES_DIRPATH}/triton_onnx_rest_servingruntime.yaml
${EXPECTED_INFERENCE_REST_OUTPUT_FILE_PYTORCH}= tests/Resources/Files/triton/kserve-triton-resnet-rest-output.json



Check warning

Code scanning / Robocop

Invalid number of empty lines between sections ({{ empty_lines }}/{{ allowed_empty_lines }}) Warning test

Invalid number of empty lines between sections (3/2)
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot Fixed Show fixed Hide fixed
github-advanced-security[bot]
github-advanced-security bot found potential problems Sep 18, 2024
View reviewed changes
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
${host}= Evaluate re.search(r"${PATTERN}", r"${host_url}").group(1) re
Log ${host}
Sleep 5s
${token}= Get Access Token Via UI single_model=${TRUE} model_name=densenet_onnx project_name=${PRJ_TITLE}

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (121/120)
@Raghul-M
Copy link
Contributor Author

Verified with Jenkins Build 397

@Raghul-M Raghul-M added verified This PR has been tested with Jenkins and removed needs testing Needs to be tested in Jenkins do not merge Do not merge this yet please labels Sep 19, 2024
@Raghul-M Raghul-M added the new test New test(s) added (PR will be listed in release-notes) label Sep 19, 2024
@Raghul-M Raghul-M requested a review from bdattoma September 19, 2024 11:54
bdattoma
bdattoma reviewed Sep 19, 2024
View reviewed changes
ods_ci/tests/Resources/Files/triton/kserve-triton-onnx-gRPC-output.json
@@ -0,0 +1 @@
{"modelName":"densenet_onnx","modelVersion":"1","id":"test1","outputs":[{"name":"fc6_1","datatype":"FP32","shape":["1000"]}],"rawOutputContents":["crbswY2eFMHy2ljBr3gZwk41nsGFaUfAwRMFwmtDScGn5h7BrHeHwdxaF8GCdt7A4DwkwezErcF98U/BQW4Iwbxw88BHxz7BmN4fwVPN7sGNiG7B5blvwUd0P8F3MqTB4RxUwbKIucHzFATCn6r7wTvnpcH5wczBxbHcwZmxvsE2KabByCZewaYnrkAMlIzBQuTfv1Mozz84gQ9B+IBkwanLXMGIUhXBIDmSwG6UfcHxIofBeiZKwTf1KEDOJ0PBAunpwdgtwMF3dCLB7/CNQDjkUEEnZu5AYr9PQCD7DUATXPlByjflwEOTasGPwhs95/zYQfriPcEGbXTBrvKpQHfmI8F/0ofBT3UfQZVHAEBHosw+kavuwUTvScGkB8a/m3UEwWWaT0B+3q3ApvsUQfTd6b/5zw5A6lbTwN7mC0Eq3wvCr6kJwovtc8DlbVjBtE8RwjkXA8JVDobBiAFfwUy7h8HLr9vBOZ5/waKBC8KYJojAJNBDwSJe6D/kgLrBVIeNwDm/z8BMv2HByQWnwFvF3sArnAnCQ0/QwSzPAcKd45PBIta0wYjICsLMab/BMNVWwb8KC8AVgiTCbRhSwSSN60BG/ePAse6rwXwZvMGfAgbCeYuTvvCQLcDvJonAP6qNwUsRFj9HFY1ARftxwaXOAsE46/rB8EmFwaBmz8FmkaPB/XuXwd+XTcEIDQbCXAW8wdX6ZcDhwQTCy3VCwWxDQcFKUszA6aYCwrWFpUDvVpjB6ondP0pUQcE9BWvAG6TYwcASsMEJa5nBsIC6waho8cGww+PB6ta6wbxBxjyvvOlAnwBXQeRlTcDGOCvA5cOpwJj/KkLaobG+18rdwDbLMkC0W4jALF6uwP5p2cBdSqbBh512wAbP1sGLRZrB2UxhwcfuGcFKOMnBPlh+wRB3p8Gl1jbBEQ+/v4NMJkGoerG+ApdbwcTZDcExhpPB7NQLwpOJ777S7IHBOLQNQUUNo79UXRrAc8wNwKkJCkG5X0BAFNUSwchLTsDHdTlAqJxSwHkrlkDtUI+8MjQ1wX4DVD5nY/xAmfPhQHht8T+0+4hBlbOEwCi+V0Gy8au9WvGtQAHNO8FG+6TBt100wV4Dk8GOEp3BxxekwT/52b92r8rB3CUmwby4LcFJdnHB0cwJQbTjv8CeYh3B82xmQAfyS0HNhSlBWaenwOTCW0AoQB3Bcd7xwIXQrUHo3ie/LGRiQXphF0EytZNBkQ8DQTASp8BfMo9AiR8iwaMbmEAkx9nAyzfyP1xFTsENAGNBzM+Tv37pjb5UCM7APYwVwW0UukCEoc5ACX+BwVWM/cBEcZxA56qHP2UiNUCSYzrBfIGswEJn1kHwkpnA501/weD5m0Bt8CzBz5r8QNNSzUB+x/s/9ymbP48J5b6ae4ZBzri0QcTDp0FCtGZBrvSlQaGtrcFRjJ7BmdHawf0zYMHKkjnB+ECGwQxM78DugSbBTOnJwZPTPsDMZMDB1QwzwbaenMFAwo9BXFwVQRrQmz/0OL/BCHB/P84i5sFUmP7AEAsswWX0hcHqmJrBfS+TwXUyicEepmVA1vS2wQOJ88GHK7/BZs/BwclFm8GRoNLBn+QDwXdfqsB+nmFAFqxNQRmHkMFU5qLBJCnOwOB7q0DQS/bAzTeDwdm0uEGFOEVBJLugQVeFtsCqObTA9YWtwKTqEMLAyprBPml2wdO0WUC/t+rA5mMbQXFtkMFeyZRBwfWywSstE8FdGsC/1+eKQRPUq7+c/ObBCZBBwNaKsMDpu9hADkbHv29S1MF/iTrBqs4TwvL5BsJd3pTBmITIwSpjZ8Flo9/Bbm20wXY3HMI29RPCpuICwup/LMI3wPXBZ9DOwUjeEMLMa/7Bz7gFwp9wfMFHxjPBIB20wTVmoMHR7KvBeEiWwQhSU8HJXwTBRmEOwvt/n8He5AvCEmkzwRFYLMFP2jjBB2jNwUhwCcHCUx/BkZ1BwS/r+8HIX+zBp2bgwBtyGMFz3uzBfjsdwrhsBsL9H4XBFm/gwCr328F+uDXBG+PVwfXgMsBxZoXB3uK7wUlJy8G5tZfBrr9WwdpAB8Kf25jBFjRTwZYhFMLJqETBSQqKwVzvgMG0h8LBQviZwaHCvMF6YfFBG99XwYO7rED1LIpAw8RDQfYE2MES+hxB+zNGQZVjV8GiYZlBWhUewdLeg0L2raXBFWJuQYfGiEFtzPA+5Vv9QLZDQEBAPNvAqOAdQhkNt0GtM2Y/0DAKwcJfiDz6aK9Bq5PjQZgApL+vrarBAM9XQWDnfkGLJypAEarxQGiMM8Fl3AVCa1cFQRak7sG81P/BAda1vyO3rUG3kZ9AtBOFQaFkj0BBB3FBcM2xQcErfUFgZpnA8hztQGiOUz+2zH9Btr0OQU0dIcF5D8nBPl0NwFnHEMHbNTXBctmqQOcoVkEZTpe/rm6oQf3lMkLBYKzB4n1PQWOmTsGL/9pAMKOjQNblqEHonttBRH0TwTtsyr9PEVjBamqFQTvAjEE58LlBySEPwc+MMsE5Jp1AtB+EwfsaQkEyvwDBhYkvwSrkG0CDGjhCel49vrQNw0DUXDRBUSHpwHhJh0An1ctBvQYLQUOvykANcZtB1qA7QVZ5xkDnAWHB69YawQXDB8HiuYJBA0SPQN78tMHHIA3BGP23QTiktUDfiIjB0iqFwReXqr8LJwBCaoNxQmRIVkLdtxxC9Cb0QJj8VUHmOUJAigJvwayBaMDxXHNBeQHaQOArPcH5hae/qLkJQnlKnsDest1BgmlkwZx2X0Gi8dVBQ+vLQTCFAUHa30C/bEbDwCJ/BEJbCRlCA9HXP57QnMHOtmVCJoeXQT9qHkJgGhs/lNXDQXrE6UEC7ZjBAoGRwZgLw0H6aXS+BQ3YwaUI+kGe48hBScYqQjw9wkDdMvJBYfOBQUmNq8FQlCtBaX2YQbwUTUJt4sxBDwSAQcIHl0Fz9TDBNTFeQbCzGkHQqZ1A6h3wvvt5L0KPAbRBA4QrwGhX1cADW9NAosItwaelbkFq6NVADy28QUEfkMBafZDBOolTQYt7K0HGhfJBFUwXQDKI7kEfBBrAlcpMwW4+vkFuS2FB8HBJQFVhQMGlU6ZB9gqAQeOtUMBztutBXfxUQVQRAUG4jG9ARCeSQVaX1b4hUZZAWkwNQSdF7UBYOb7ALOyPP8YQScGvYbdACM+bwYfXwUEPd8lBLp/RQdnsWEHwDcNAOCTiwewGx0G2VhtCSYSLvzaHIr/ybLZATErGQc9jE0GQ3cs/EH7WQGaGEkEkcra/R9J4Qau7AkIgKx9B3VH6QFOPcEKp3nVBWTxMwaXdl0C7BM3AjxejQdpShcE73zBBThCcQdPZ28FKZDZBVJMBwfualEG7WEBCCLNWQaDgEsKZ9b9BqbYeQT7PGcC1a+hBGEX2wH7NpMEAnhhCq9yxQQfWAkHXJ9NBoAQvvgcs1UFtvsZBR8bswOr5C8F8mwFCNsbHQaS2xsAWWzrBakuBQYEmIkF1uyBA1kIZQcIQpUHjLqxB11u2wdDaGEGki2rBqQvowFKE/UEF3uTA/QlOQbfkREHF1qnAcfJSwfn9JEHNnAW/em6hQLmFvEHho/hAhGgGwrxrskGMrYdAq+dJPqW0ikEI1KhBNEIGQnMRU8FS+YxBPWttQFaPhEFDiB0/KnFAwed2ocDA6pLA2/j6wVup3762GYBBzCsYP2mouL/+DYM+ifhJwExf3cApzf/Ad6imQHPbaMDP3qlBWtGwwPB2RLxF8MQ/z1edPwelI0FAOb7AooLSQCTktcAEG+tBE6KHQe6s6sAForFB2UikQPlGAsIRfDrBgw2/QVpveUG6WAQ/uC4kQWlphEG8ZlNCVlA3QoLjZMECaQRCqrgewRwT20H8iBzBZCHcQYkcvsDs+BVBKRO5Pnq13EHsvGNB1JHXwMqe8UHmsEBB+kcgwK+nukC0giPBgSumwd7f2EEZSVfBf5IwQfsLmkFQs5ZB24+6QaRrtcHkLLM/L4jVPpzdc0EnNt9A05X6wJyRT0IZe+S/++BfwRv3JsAaKppBA+MoQTSR08EF1NtBJn8dQRmRAMLHw8vAwRgIQkc6A0L2PgvAtHKnQV0/m0HgNDLBpomjvxt1eT/eovVB6YYPQdx4XsGZrR5Ah7vBwL5Zt0GfdYNADTknwWzvBkJHyhNCrmB9QYycWsGf94NBKJ8GQfZSuUHyNjHALCVfQS9jyEBaKmBB9PeuQfytnMHmA2vAqiOPQdwq1MB6DCfApxEDwVuxP0FgqKRByOkUQDtUPsF3HUPASOiHQRsLdEEu8JG/fk+eQcgZG78hyFxBnvP+P9u3hkE2CxLBh6moQW7lJcFvYgRBvgyiwMaYlUAblTtCcEYEQfXyXsHC6oFA7QrSQfCLokEdI19B4F/ywMjGyUG1RPhB6NDIQYXc+MBE7idBzFMtQcZtRMHG7JbBEEp9wUOI28GxW/1BZ/10QVdqfkEbsaBA8H28QYEzg0BsYi9BlmH0QArmQ0F99D5Bcu84Qq6mvMHDENBAaqJqQtEfOEF6EhlC60T8QSHrQkAcDtJA0yKrQCTNAcFA2FPAVTKkwbfrK0Lqs+BAogAev5E7DkFiBInARb0bQaMw0D/FrhfBn6dnwSlKOULEP5TAnYCkQXvhFcGkDrVBdn0fQFGpqsGYLDBBp6VxwRKY0EFBFo/BZ6WlQYSpgsAaAKPAi164QXWXlUFE76g/k1uhwUpLW0HuHXTB+IstQfpDzUC9XW/ACTA9QclM8kGxhwzBDwQTQRLxk0ELp4BBqc3LQQPHFEKkxiZCKy7RQPK6AcF0HCNCwHm7QfOCQ0E1ylFBBTgiwV7njEGt8rVB9abWQV4XzT/6Fu5AXX0kwedACML+Ev3BUKUGQC7KQULFOA9BMpncQb/n40FiF9pBwQclwRE6UkH+mjRBWJalQMcx/EArevXAODoLwXKIVUFfzYBAO1nWQG0+kEEjWqhBtsuvwD+qCEFhaRZA7nn6voqpwEHRZBZBt89Dv1JEOMFfYavB1u4tQO5TlD8xF1vA7k2ZvpIQG8FB629B6KemQVoelUFl67VBfZqKQU6UV0EAhBHBNCwEQin06UBfib1AsfsGQbsUb8BKyAlBSaN0QW63zcDjTAhBvaAGwZD+RkAOGxvBKlIIQo1aLEJMy39CqhEAQoOoxsCtt+NADdXiPymzDMGoeoDBnjwJwYaWQcF4GShB+8sdvzs1JcGnHjHByDiAwBwcl0EWbVvAegd1wO9soEG9BwrCBmWiQd2Bib+8jYjBw2Jhv5Adu8G4BIHBv0z5wWZ3BcKElV/BMe6wQAPnvcGmp5JAqHhTwQ=="]}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't the input/output the same as the ones used in REST test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's different

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? because it is using a different model? otherwise I don't expect input to change based on the protocol

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The input is different, so the output is also different for REST and gRPC

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah ofc. I meant to say that I was expecting the input (and then the output ofc) for the same model to be the same regardless the protocol

bdattoma
bdattoma reviewed Sep 19, 2024
View reviewed changes
ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDashboardSettingsRuntimes.resource Outdated
@@ -100,5 +100,6 @@ Select Runtime API Protocol
[Documentation] Selects which API protocol for the runtime
[Arguments] ${protocol}=gRPC
Click Element xpath://button[@data-testid="custom-serving-api-protocol-selection"]
Wait Until Page Contains Element xpath://li[@data-testid="dropdown-item ${protocol}"]
Click Element xpath://li[@data-testid="dropdown-item ${protocol}"]
Sleep 10s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try not to use sleep where possible. You would add timeout in the next wait keyword

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replaced sleep with timeout

@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

@Raghul-M Raghul-M requested a review from bdattoma September 19, 2024 13:40
@Raghul-M Raghul-M removed the request for review from lugi0 September 19, 2024 14:51
@CFSNM CFSNM merged commit 2b2b2ed into red-hat-data-services:master Sep 19, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bdattoma bdattoma bdattoma approved these changes

@CFSNM CFSNM CFSNM approved these changes

@tarukumar tarukumar Awaiting requested review from tarukumar

Assignees

@Raghul-M Raghul-M

Labels
new test New test(s) added (PR will be listed in release-notes) verified This PR has been tested with Jenkins
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Raghul-M @CFSNM @bdattoma