Insufficient permissions on volume for non-root containers

[sereinity]

What happened:

Pods crash at start because it can't create files in the new PVC.

I'm able to make the pod work by manually create a temporary pod, mounting, create files and changing ownership.

What you expected to happen:

I expect that the new CSI provisionner can work as a drop-in replacement of the kubernetes.io/azure-disk provisionner.

How to reproduce it:

Create a new cluster with aks version 1.21.1.

loki:
  persistence:
    enabled: true
    size: 15Gi
    storageClassName: default # switch to "managed" to make it work

helm upgrade --install --create-namespace --version 2.4.1 -n logging -f values.yml loki-stack grafana/loki-stack

Anything else we need to know?:

Environment:

• CSI Driver version: v1.2.0 and v1.4.0
• Kubernetes version (use kubectl version): Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"ca992926ba9f76abd853859a64b5b04a1b562362", GitTreeState:"clean", BuildDate:"2021-05-19T22:29:12Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools: helm v3.6.0
• Others:

[edreed]

I was able to reproduce the issue with the deploy/example/statefulset-nonroot.yaml example. I found a related issue in kubernetes-sigs/vsphere-csi-driver#370 and tried the workaround of explicitly setting the fstype parameter in the StorageClass to ext4. This worked for me. @sereinity Can you give this a try and report back whether this works for you?

[edreed]

/assign

[andyzhangx]

this issue could be related to fsGroupPolicy, which is only supported from 1.20, and you need to run following command to support fsGroupPolicy: https://github.com/kubernetes-sigs/azuredisk-csi-driver/tree/master/deploy/example/fsgroup#option2-enable-fsgrouppolicy-support-on-a-cluster-with-csi-driver-already-installed

When minimum supported k8s version is 1.20, we will enable fsGroupPolicy feature by default.

[edreed]

@andyzhangx Thanks! Enabling fsGroupPolicy also worked for me.

[sereinity]

Thank you @edreed , @andyzhangx I confirm that both enabling fsGroupPolicy or explicitly setting fstype worked for me.