Support passing labels and annotations in ServingRuntime to runtime Pod

[lizzzcai]

Is your feature request related to a problem? If so, please describe.

We have a custom webhook to validate and mutate the actual pod created by the ServingRuntime based on the labels/annotations, for example, mutate resources limit and validate user info added in the annotations.

However, currently the ServingRuntime will not pass its own labels/annotations to the deployment/pod and there is no field in the spec to provide labels/annotations as well. The runtimePodLabels and runtimePodAnnotations in model-serving-config are system-wide configurations but our use case is setting the value per ServingRuntime.

Describe your proposed solution

1. pass labels/annotations in the ServingRuntime's metadata to the deployment spec.template.metadata, or
2. add labels and annotations fields in the ServingRuntime spec and pass them to the deployment spec.template.metadata.

Describe alternatives you have considered

Additional context

[njhill]

@lizzzcai this sounds very reasonable and would be pretty trivial to implement. We should definitely follow the second proposal though, specifically to add annotations and labels fields to the ServingRuntimePodSpec.

This would apply similarly to single-model runtimes so would also require a small update to the original kserve controller.

PRs are very welcome!

[lizzzcai]

Hi @njhill , I have created one PR in kserve controller, feel free to review it when you have time.