Add Design for Dynamic export of ConfigMap/Secret values

[matthchr]

Related to #2555.

If applicable:

• this PR contains documentation
• this PR contains tests
• this PR contains YAML Samples

[theunrepentantgeek]

I think it may be better to pass in spec and status as the two available variables, since everyone will need to index into one or the other first. This would allow us to also add secrets as a third variable, which is only included if secrets. exists as a substring in the expression.

[matthchr]

There's also metadata though so I think just passing the whole top level object might be cleaner?

[theunrepentantgeek]

I think the tension here is between making things possible and making common things easy.

Passing just the top level object makes things possible but it forces every usage to incur a little bit of overhead. Passing multiple objects is more complex to implement, but has the benefit of making most usage simpler.

[matthchr]

Still not sure I agree. resource.spec.whatever isn't much more complicated than spec.whatever.

The advantages to us using top level names we choose are:

1. It disambiguates between the resource and its secrets, since for secret usage we'll support secrets.myKey. If we control the top-level name of all the inputs (the main 2 right now being resource and secrets), then we can ensure there are no collisions because we own the top of the path for each input. If we instead snip that part out (for just resource? or for secrets too?) then we run the risk (though it's small) that a top level field such as secrets collides.
2. It puts us in a better position to grow in the future by adding other top-level types going forward, such as operator (holding operator configuration?) or cluster (holding cluster details?). I don't know if we're ever going to want to add these but if we did, disambiguating between them at the top level helps with clarity.
3. It means we're consistent between resource.x and secrets.y in terms of the pattern, as opposed to doing x and secrets.y. I think including secrets as a prefix for secret values which are not on the resource is very useful from a security/safety point of view because we're making it really obvious the value you're about to output is a secret. Obviously we only support this in the context of saving to secrets so it's not like there's risk of exposing a secret value as plaintext but still the clarity seems a win for something sensitive like secrets.

I'll update the document to discuss this and leave it as an option question at the end of the document.

[theunrepentantgeek]

If it must be implemented alongside - that is, if it's mandatory - why not extend the existing KubernetesExporter interface with a second method, to force this.

[matthchr]

Not every KubernetesExporter produces secrets. If all it produces is configmaps we don't need to have the secret specific one

[matthchr]

I've clarified this wording some and pointed out we could add the method to the existing interface, but I think a separate interface will ultimately be cleaner. Not married to making a decision here though we can also discuss during implementation.

[super-harsh]

Would we be able to export collections? Most preferably Maps since we don't support exporting slices yet.

[matthchr]

Yes, though the user might need to write a CEL expression to explain what exactly they want to gather from the map

[matthchr]

I will update the documentation here to be explicit about how this might look in an actual CEL expression, but it is doable for both maps and arrays

[matthchr]

Updated the FAQ documentation. It now says:

Q: Can this be used to export a particular item from an arbitrary array or map?

A: Yes! The syntax for maps is: request.myMap['hello'], and for arrays request.myArray[0]. CEL has a concept of
macros and there are some useful ones such as filter, exists, and all
built in, so you can craft more complex
expressions like request.slice.exists_one(i, i.name=='foo') ? request.slice.filter(i, i.name == 'foo')[0].value : 0
which checks if the slice contains exactly one element whose name is foo and if so, gets that elements value,
otherwise returns 0. A similar construct can be done for maps using the same exists_one and filter macros.

[matthchr]

Ah, but what about if you want the whole collection..., let me think on this.

[matthchr]

Ok we can support whole collections too. Here's how:

Q: Can this be used to export entire collections of secret or configmap values?

A: Yes. We will support two flavors of entry into the dynamicSecrets and dynamicConfigMaps collections. One
flavor is the one we've seen above, where value is a CEL expression that returns a string and the name and key
fields specify which configMap or secret and what key within that to export.

The other kind of entry is:

 operatorSpec:
   dynamicConfigMaps:
     - name: account-data # Name of the destination configMap
       valueMap: "{"key1": "foo", "key2": "bar"}" # CEL expression that returns a map[string]string

The valueMap fields result can be directly saved as keys + values in the resulting configmap or secret, so the author
of the CEL expression has direct control over what values go into their secret/configMap.

Great question @super-harsh.

[theunrepentantgeek]

/ok-to-test sha=ba97db2

[matthchr]

/ok-to-test sha=f201536