GPU sharing in AKS

[shiva-appani-hash]

hi Team,

Currently, we have below restrictions on GPU allocation:

1. Containers (and Pods) do not share GPUs. There’s no over-committing of GPUs.
2. Each container can request one or more GPUs. It is not possible to request a fraction of a GPU.

Do we have any solution in place to enable GPU sharing in AKS.

[palma21]

Yes, multi-instance GPU or fractioned GPUs are available on AKS
https://learn.microsoft.com/en-us/azure/aks/gpu-multi-instance

Note that Azure only supports fractioned GPUs in GPUs where NVIDIA supports that capability officially.