Plugin Isolation

[huseyinbabal]

We are going to introduce file system isolation to prevent other plugins from using default Botkube SA.

In this case, Botkube app needs to use root permissions inside the container, which could be a security concern for some. We accept this a less evil.

This will prevent accidental bugs in plugins:

• plugins won’t be allowed to read e.g. Slack and Discord credentials
• Plugins won’t be allowed to use Botkube Pod ServiceAccount

LIMITATION: Plugins will use their own kubeconfig with user and groups, but they can still alter the Kubeconfig to use different user or group. As a result, you need to anyway trust the plugins that you install.

AC

• introduce a plugin isolation system as described in the proposal here.
• Update the documentation and state clearly, that:
  • Botkube uses impersonation
  • How plugins are isolated, and what are the limitations
  • User needs to trust each plugin as they can alter the kubeconfig and use different permissions
  • Consider describing it in a “Security” doc and link from RBAC docs
  • Nice to have: diagrams
• Add e2e tests using echo plugin. It can be renamed. However, the idea is to run bash command through the echo executor to check that we get denied error.
• Fix bugs that will be uncovered by added isolation:
  • [Bug] Kubectl executor - kubectl command builder doesn’t use the kubeconfig use kubeconfig from exec input #1060
  • [Bug] GitHub executor - ignore context.rbac at all and runs the kubectl commands directly. use kubeconfig for gh plugin #1062

Research Result
In this research, I created dedicated folders for each plugin and downloaded the binaries there to do proper chroot operation. I tried use chroot with plugin cmd, but no result due to;

• Hashicorp Go Plugin has an abstraction on plugin execution, so even you have cmd with chroot settings, it gives an error

• Even you somehow fork the Hashicorp Go Plugin repo and do chroot, you need to copy all basic executables from the root, to the chrooted directory. Assume you built a tree plugin that shows directory in tree format and it needs ls command. For this case, you need to do cp /bin/ls /tmp/botkube/tree/bin/ls for tree plugin since it uses builtin ls command. This is not flexible especially once you can implement different kind of plugins that needs different dependencies.

Is CMD without Chroot a Real Concern?
I don't think so, even Terraform itself does not use chroot as you can see here, we don't need to spend too much effort on this. In same way, as an example, Akamai provider has direct access to read env vars as you can see here.

What is the output?
We can omit chroot support, but there is still one more item which is changing the default k8s token to prevent random access by plugins.