OpenVINO Backend 2022.3 support

[riZZZhik]

Hello,

Currently triton-inference-server/openvino_backend is based on OpenVINO 2022.01
When will official Triton Container support 2022.3 or later OV backend?

Based on Changelog (2022.02, 2022.03) I assume that update won't cause any issues

Thanks!

[nv-kmcgill53]

cc: @Christina-Young-NVIDIA @dzier

[riZZZhik]

Any updates about this issue?

[GuanLuo]

@tanmayv25 can you estimate this effort?

[tanmayv25]

The standalone openvino version in OV backend is dependent upon the version of openvino released in onnxruntime version being shipped with Triton. See here for more information: https://github.com/triton-inference-server/server/blob/main/build.py#L64-L67

The actual update is not expected to be a lot of work.

[riZZZhik]

@tanmayv25 Hello,

Could you please let me know when it will be implemented

[tanmayv25]

@pranavsharma Does the latest onnxruntime release in Triton ORT v1.14.1 support openvio 2022.3 ? If yes, then we can try upgrading the openvino version in Triton.

[pranavsharma]

@pranavsharma Does the latest onnxruntime release in Triton ORT v1.14.1 support openvio 2022.3 ? If yes, then we can try upgrading the openvino version in Triton.

yes. https://onnxruntime.ai/docs/execution-providers/OpenVINO-ExecutionProvider.html#requirements

[siyuch-fdu]

We also need the 2022.3 version of openvino_backend, and hope that the official version can be implemented as soon as possible, thank you very much!

[nv-kmcgill53]

CC: @nnshah1

[nikolsky]

It would be great to see this feature in the future releases.
We have to use OpenVINO Model Server instead of Triton without it.

[pranavsharma]

The latest version of ORT uses 2023. Does that work? See https://onnxruntime.ai/docs/execution-providers/OpenVINO-ExecutionProvider.html#requirements

[riZZZhik]

@pranavsharma it supports OpenVINO 2023, but uses the one that is installed in triton server
Currently it is 2022.1 as in https://github.com/triton-inference-server/openvino_backend

[nnshah1]

Quick update - the team is currently investigating the upgrade.

[riZZZhik]

@kthui Thanks! ❤️❤️❤️