flux diff ignores kustomize.toolkit.fluxcd.io/ssa annotations

[YvanGuidoin]

Describe the bug

We are using MetalLB and have added some annotations to BGPPeer CRD to avoid this issue of FluxCD overriding fields all the time (this is not Flux fault but this bug itself seems unrelated)

CRD with annotations:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    kustomize.toolkit.fluxcd.io/force: Disabled
    kustomize.toolkit.fluxcd.io/ssa: IfNotPresent
  generation: 2
  labels:
    kustomize.toolkit.fluxcd.io/name: mc-kube-infra
    kustomize.toolkit.fluxcd.io/namespace: flux-system
  name: bgppeers.metallb.io
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: [...]

Running

$ flux diff kustomization mc-kube-infra -r --path test/ --progress-bar=false
► CustomResourceDefinition/bgppeers.metallb.io drifted

metadata.generation
  ± value change
    - 2
    + 3

spec.conversion.webhook.clientConfig.caBundle
  ± value change
    - LS0tLS1CRUdJTiB...(jwt)...LS0tCg==
    + LS0tLS1CRUdJTiB...(different jwt)...LS0tLQ==
  

kustomize-controller behaves properly and ignores the field, avoiding noise on apply:

{
    "level": "info",
    "ts": "2025-01-29T18:38:13.923Z",
    "msg": "server-side apply for cluster definitions completed",
    "controller": "kustomization",
    "controllerGroup": "kustomize.toolkit.fluxcd.io",
    "controllerKind": "Kustomization",
    "Kustomization": {
        "name": "mc-kube-infra",
        "namespace": "flux-system"
    },
    "namespace": "flux-system",
    "name": "mc-kube-infra",
    "reconcileID": "c4217ca1-1b84-4da6-8def-4532440ede31",
    "output": {
        //...
        "CustomResourceDefinition/bgppeers.metallb.io": "skipped",
        //...
    }
}

Steps to reproduce

Deploying MetalLB with Flux:

---
namespace: metallb-system

resources:
  - "https://raw.githubusercontent.com/metallb/metallb/v0.14.9/config/manifests/metallb-native.yaml" # tag=v0.14.9

patches:
  - target:
      group: apiextensions.k8s.io
      kind: CustomResourceDefinition
      name: bgppeers.metallb.io
    patch: |
      - op: add
        path: "/metadata/annotations/kustomize.toolkit.fluxcd.io~1ssa"
        value: "IfNotPresent"
      - op: add
        path: "/metadata/annotations/kustomize.toolkit.fluxcd.io~1force"
        value: "Disabled"

Expected behavior

flux diff should skip resources with kustomize.toolkit.fluxcd.io/ssa configured

Screenshots and recordings

No response

OS / Distro

Ubuntu 24.04

Flux version

v2.4.0

Flux check

N/A

Git provider

GitLab

Container Registry provider

No response

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[YvanGuidoin]

Looking at the code to see if I could solve it myself, this might come from https://github.com/fluxcd/flux2/blob/main/internal/build/diff.go#L100 which skips objects flagged with kustomize.toolkit.fluxcd.io/reconcile: disabled only.

For kustomize.toolkit.fluxcd.io/ssa they are 2 cases Ignore could skip the same as reconcile: disabled, but IfNotPresent should skip only for the case existingObject != nil in https://github.com/fluxcd/pkg/blob/main/ssa/manager_diff.go#L60
This is kinda already the case but not in a clear and declarative way.

DiffOptions in https://github.com/fluxcd/pkg/blob/main/ssa/manager_diff.go#L33 uses a map[string]string, which doesn't allow multiple values for one annotation...so no quick solution that I can think of

Edit: think I found out to do it, but will require a change also on fluxcd/pkg/ssa

[YvanGuidoin]

With fluxcd/pkg#862 merged, waiting for new version of fluxcd/pkg/ssa to change code in

flux2/internal/build/diff.go

Line 100 in e09ba7a

diffOptions := ssa.DiffOptions{

to

diffOptions := ssa.DiffOptions{
	Exclusions: map[string]string{
		"kustomize.toolkit.fluxcd.io/reconcile": "disabled",
		"kustomize.toolkit.fluxcd.io/ssa":       "ignore",
	},
	IfNotPresentSelector: map[string]string{
		"kustomize.toolkit.fluxcd.io/ssa": "ifnotpresent",
	},
}

to align to kustomize-controller behavior
https://github.com/fluxcd/kustomize-controller/blob/59676860b8cb5bb25db94efc7dea8b75f5966998/internal/controller/kustomization_controller.go#L678

[stefanprodan]

waiting for new version of fluxcd/pkg/ssa

Just got merged here #5175