Support running multizone global and zone CPs in the same k8s cluster

[bdecoste]

Summary

Id like to be able to deploy multizone global CP and remote CP to the same k8s cluster

Steps To Reproduce

1. Deploy global multizone CP to a k8s cluster
2. Deploy remote multizone CP to the same k8s cluster. This is not possible as of 1.0.6

Additional Details & Logs

• Version
• Error logs
• Configuration
• Platform and Operating System

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[coreywagehoft]

I would like to see Kuma support this so that we don't have to operate a completely separate Kubernetes cluster for the global control plane.

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[lahabana]

xrefs:

• Global DPs are created in default namespace  #3247
• Support Kubernetes Namespaces in Policies #2921
• Service name from K8s breaks encapsulation and is inconsistent with universal mode #2780
• Specify Mesh via label on K8S #738

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[lahabana]

This is planned but we first need to move all the policies to be namespace scoped.

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[lahabana]

Policies still needing to be migrated:

deployments/charts/kuma/crds/kuma.io_externalservices.yaml: (talked about removing tags for example or at least do something better here).
deployments/charts/kuma/crds/kuma.io_meshes.yaml:
deployments/charts/kuma/crds/kuma.io_meshgatewayconfigs.yaml: --> Required by GatewayAPI shouldn't change
deployments/charts/kuma/crds/kuma.io_meshgateways.yaml:
deployments/charts/kuma/crds/kuma.io_serviceinsights.yaml:
deployments/charts/kuma/crds/kuma.io_virtualoutbounds.yaml:
deployments/charts/kuma/crds/kuma.io_zoneinsights.yaml:
deployments/charts/kuma/crds/kuma.io_zones.yaml:

• All our configMaps/Secrets are namespace scoped already.
• Need to figure out a way to deal with webhooks. (Does a namespace have a zone reference?) -- Only needed to run multiple zones in the same cluster.
• Tests

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[lahabana]

Random thoughts we had with @jakubdyszkiewicz :

• We can do so that KDS figures that the synced cluster scope resources are already in the cluster (using uid for example, or just not fetching them with an explicit flag)
• We can skip reconciling policies in other namespaces annotated with kuma.io/system-namespace: true
• Webhooks remains a question or maybe it just relies on system-namespace too or global has a webhook only for its system namespace.

The only reason for not having mesh cluster scope seems to be able to do this, which in the end might possible even with cluster scoped policies.

Though you'd only be able to run 1 zone per cluster. To be able to run multiple zones you'd probably need to say which zone your namespace is managed by (with an annotation for example).

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[slonka]

Isn't this partly solved by #5907 ? The original issue did not say that both of them have to be backed by k8s storage.

[jakubdyszkiewicz]

Triage: it's about deploying global and zone cp with k8s storage

[github-actions]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant, please comment on it or attend the next triage meeting.

[lahabana]

Is this something people still ask for?

[slonka]

Triage: closed, let's reopen if needed