Add ability to store sops metadata in OpenAPI compliant format

[isindir]

This is openining discussion and/or potential request to:
Implement sops metadata in OpenAPI compliant format.

Problem description:
I've written Kubernetes sops-secrets-operator which allows to use sops as cli to encrypt Kubernetes CR definition for storage inside git repository for GitOps using FluxCD. We found sops very easy to use inside the team. The issue is that all Kubenrnetes resources (including custom resources) must be OpenAPI compliant. From k8s version 1.15 non-compliant fields will be removed from resource at creation time in k8s and not stored in etcd. However operator relies on information passed via sops metadata, which is not OpenApi compliant - ie fields like gcp_kms, azure_kv, encrypted_suffix, created_at, aws_profile, etc. That makes it impossible to continue development and use of the operator as it will not be compattible with k8s version 1.15.

Questions about feature implementation:

• Is it easy to implement extra flag to sops to store metadata in OpenAPI compliant format (using existing format by default) ?
• Would it be easy to detect at decryption time the format used ?

I'd be happy to help with this feature development, but will need some help and guidance.

Thanks

[autrilla]

I'm not 100% sure I understand what you're asking, but I think this is related to #401. There are more ideas about the possible interface SOPS could expose to users, and as possible users, your team's opinion on those ideas would be helpful 🙂.

My opinion is that Kubernetes is big enough that a separate subcommand for related operations could make sense, even if it's just a thin wrapper over a more "generic" Kubernetes store. In short, I think this is definitely something we want in SOPS.

[isindir]

#401 is definetely related issue. I've created k8s operator (added link above), which can be used at the moment with sops command as is, by encrypting part of the file (see example bellow). The problem is - sops metadata uses snake_case for field names, where OpenAPI standard has restriction on lowerCamelCase for field names only. The operator uses sops as library to decrypt custom resource and create k8s secrets from that definition. As of now everything works fine except that I can't create CRD (custom resource definition) for k8s, which would list all the fields in metadata correctly and check existence of those. In version 1.15 of k8s such non-compliant fields of the Custom Resource will be trimmed and not applied to the etcd as these are not defined in CRD.
Please see following link for more information: structural-openapi . What I'm asking here is an ability to store metadata field names in lowerCamelCase format via sops command line utility by specifying extra flag. That would allow to use sops command as is to edit CRs without wrapping it.

I would still need to do changes to the operator (I have none compliant field - _template in the definition.

The fields I mentioned are from example bellow:

apiVersion: isindir.github.com/v1alpha1
kind: SopsSecret
metadata:
    name: tst-secrets
    namespace: tst
spec:
    secret_templates:
    -   name: ENC[AES256_GCM,data:<redacted>,iv:<redacted>,tag:<redacted>,type:str]
        data:
            MY_PASS: ENC[AES256_GCM,data:<redacted>,iv:,tag:<redacted>,type:str]
    -   name: ENC[AES256_GCM,data:<redacted>,iv:<redacted>,tag:,type:str]
        data:
            MY_PASS2: ENC[AES256_GCM,data:<redacted>,iv:<redacted>,tag:,type:str]
sops:
    kms:
    -   arn: arn:aws:kms:<redacted>:<redacted>:alias/<redacted>
        created_at: '<redacted>'
        enc: <redacted>
        aws_profile: ""
    gcp_kms: []
    azure_kv: []
    lastmodified: '<redacted>'
    mac: ENC[AES256_GCM,data:<redacted>,iv:<redacted>,tag:<redacted>,type:str]
    pgp: []
    encrypted_suffix: _templates
    version: 3.3.1

[autrilla]

I see, thanks for clarifying. I can't see a way to do this cleanly right now, but I'll see if I come up with something.

Implementing a separate Store for this won't be clean. I though maybe it could be a simple wrapper around the yaml store, but we hardcode the field names for the emitted YAML here, and changing them globally is not an option as it would break backwards compatibility. So we'd either need to maintain an exact copy of that struct (with the field names changed) in the hypothetical yaml/openapi store, or have the store process the YAML generated by the yaml store and add or remove the camelCasing as needed. The former will make maintaining SOPS harder as we'll need to keep both structs in sync. The latter needs to be done recursively over the whole sops: YAML branch. I'd lean for the latter, even though it's more complicated to implement, it'll be isolated in its own package and it won't make maintaining the rest of SOPS harder.

[autrilla]

I am also not sure whether implementing this would be optimal. It would perhaps be better to reach agreement on what a good solution for Kubernetes workflows would be and implement that for everyone, instead of creating a new Store that will mainly work for your particular operator. It would be good to check that this will be useful to other potential users.

[isindir]

I think latter would be better option.

I think the main two ways of deploying secrets to kubernetes are - via CI/CD pipeline of some sort and via gitops CD. If done via CI/CD pipeline, sops is sufficient as is. But to incorporate into gitops CD an operator is necessary and to simplify secrets maintenance it would be handy to be able to use sops without wrappers of some sort. It is proven really easy to use the operator - by encrypting part of the CR file using sops and pushing it to git repo via PR.

[StevenACoffman]

Not ideal, but the operator could pass only a small blob of CRD data to sops-as-a-library to decrypt into a kubernetes secret which would be encrypted at rest in 1.15.
An sops-operator CLI could take a sops file and wrap it into the CRD format (like fluxctl, or istioctl, etc.).

Alternatively, you could sanitize the relevant portions and translate the keys into and out of camel case using a library like strcase or conjson or using a technique like in this stackoverflow question.

You could also create kubernetes specific camel case variants and trivially convert into the regular sops ones. Just throwing out ideas. thanks for cool stuff so far!