Updating SOPS to 3.7.1

Sops 3.7.1 introduces a limited AGE support[1]
Disabling gpg importing key debug by default (can be overridden)

[1]
https://github.com/mozilla/sops#encrypting-using-age

examples/contrib/sops/README.md

@@ -2,45 +2,28 @@
 
 The `sops` config function transforms an input kpt package according to the
 function configuration: in the current version it can encrypt yaml documents or
-decrypt the documents that have `sops` field with SOPS metadata.
-This example invokes the `sops -d` for decryption and `sops -e` for encryption
-function using declarative configuration. See
-[sops readme](https://github.com/mozilla/sops/blob/master/README.rst) for more
-details.
-
-In order to encrypt or decrypt yaml, `sops` may accept a variety of ENV vars, e.g. to work
+decrypt the yaml documents that have `sops` field with SOPS metadata. What exactly
+operation will be performed is set by `cmd` field of the function configuration.
+Depending on that field value the function invokes `sops -d` for decryption and 
+`sops -e` for encryption. The function passes all other parameters directly
+to the command-line parameters of `sops` tool: e.g. if it's necessary to pass
+`--unencrypted-regex <value>`, it's possible just to add a field
+`unencrypted-regex: <value>` to the function config. 
+See [sops readme](https://github.com/mozilla/sops/blob/master/README.rst) for more
+details on the parameters and environment variables that `sops` tool accepts.
+
+There is an option to specify only a particular subset of input yaml documents that have to be
+encrypted or decrypted: `cmd-json-path-filter` field accepts JSONPath notation to do so.
+E.g. `cmd-json-path-filter: '$[?(@.metadata.name=="somename" &&@.kind=="somekind")]'` will process
+only documents with name 'somename' and kind 'somekind'.
+
+Another special option is `cmd-tolerate-failures: true` that ignores the `sops` tool errors on
+any operation and just keeps the document unmodified in that case.
+
+In order to encrypt or decrypt yaml, `sops` may accept a variety of ENV variables, e.g. to work
 with Hashicorp Vault it will be necessary to set: `VAULT_ADDR` and
-`VAULT_TOKEN`. For PGP case this function introduces an additional `SOPS_IMPORT_PGP` ENV var
-that must contain the private key or keys needed to decrypt yamls and public key or keys to
-encrypt yamls. If you have a file with keys it's possible to run:
-
-```sh
-kpt fn run --env SOPS_IMPORT_PGP="$(cat <file with exported key>.asc)" <folder>
-```
-
-or if your keys are already in `gpg`, it's possibe to run:
-
-```sh
-kpt fn run --env SOPS_IMPORT_PGP="$(gpg --armor --export-secret-keys)" <folder>
-```
-
-## Function invocation
-
-Get this example and try it out by running the following commands:
-
-```sh
-# download sops kpt-function example
-kpt pkg get https://github.com/GoogleContainerTools/kpt-functions-catalog.git/examples/sops .
-# copy example key from sops project
-curl -fsSL -o sops/key.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc
-# run the function
-kpt fn run --env SOPS_IMPORT_PGP="$(cat sops/key.asc)" sops/local-configs
-```
-
-## Expected result
-
-Verify the updated configuration:
-
-```sh
-kpt cfg cat sops/local-configs
-```
+`VAULT_TOKEN`. This option can be used to set different encryption parameters that shouldn't be stored
+in git repository, e.g. private keys, external services credentials.
+This function introduces 2 additional ENV variables: `SOPS_IMPORT_PGP` and `SOPS_IMPORT_AGE` that
+make possible to work with PGP and AGE encryption. Please refer to [gpg](gpg/) and [age](age/) examples
+to get more details.

examples/contrib/sops/age/README.md

@@ -0,0 +1,41 @@
+# SOPS AGE example
+
+SOPS function introduces for AGE case an additional `SOPS_IMPORT_AGE` ENV variable
+that must contain the [SOPS age keys.txt file](https://github.com/mozilla/sops/blob/master/age/keys.txt).
+This file is needed for decryption. For encryption it's possible to use
+`SOPS_AGE_RECIPIENTS` ENV variable or parameter `age`.
+
+E.g. for decryption it's possible to run:
+
+```sh
+kpt fn run --env SOPS_IMPORT_AGE="$(cat <file with age keys>.txt)" <folder>
+```
+
+or if AGE keys are already stored in the host system so SOPS binary works locally, it's possible to run:
+
+```sh
+kpt fn run --env SOPS_IMPORT_AGE="$(cat ~/.config/sops/age/keys.txt)" <folder>
+```
+
+## Function invocation
+
+Get this example and try it out by running the following commands:
+
+```sh
+# download sops kpt-function example
+kpt pkg get https://github.com/GoogleContainerTools/kpt-functions-catalog.git/examples/contrib/sops/age .
+
+# copy example AGE key from sops project
+curl -fsSL -o age_keys.txt https://raw.githubusercontent.com/mozilla/sops/master/age/keys.txt
+
+# run the function to work with AGE
+kpt fn run --env SOPS_IMPORT_AGE="$(cat age_keys.txt)" age
+```
+
+## Expected result
+
+Verify the updated configuration:
+
+```sh
+kpt cfg cat age
+```

examples/contrib/sops/age/function.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: my-decrypt-config
+  annotations:
+    config.k8s.io/function: |
+      container:
+        image: gcr.io/kpt-fn-contrib/sops:unstable
+        envs:
+        - SOPS_IMPORT_AGE
+    config.kubernetes.io/local-config: "true"
+data:
+  cmd: 'decrypt'
+  cmd-json-path-filter: '$[?(@.metadata.name=="toDecrypt")]'
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: my-encrypt-config
+  annotations:
+    config.k8s.io/function: |
+      container:
+        image: gcr.io/kpt-fn-contrib/sops:unstable
+    config.kubernetes.io/local-config: "true"
+data:
+  cmd: 'encrypt'
+  cmd-json-path-filter: '$[?(@.metadata.name=="toEncrypt")]'
+  age: 'age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw'
+  unencrypted-regex: '^(kind|apiVersion|group|metadata)$'
+

examples/contrib/sops/age/to-decrypt.yaml

@@ -0,0 +1,27 @@
+apiVersion: v1
+kind: somekind
+metadata:
+    name: toDecrypt
+nnn-password: ENC[AES256_GCM,data:G2i53KNBoAEycMHD5w==,iv:PjywQXtBGZDgn5zNgI/0BsKiov7drduyyXSt8gNtdDs=,tag:slGGtNbvT01rtgDb1pCkFw==,type:str]
+user-password: ENC[AES256_GCM,data:iM5w/l6egeDlvkjc2tpk,iv:kBtgkTNuNuUL6FvYGa+NA0U0jkcmm0IIRZ8j5PicSMs=,tag:lACKq9kbf4n72xFmZb1Ymw==,type:str]
+k8s-password: ENC[AES256_GCM,data:uZb3qTvx3Ucq,iv:VQTP3fimb3uitY/0LuJMhvdjd2s9bq8xguWgTOkCqG4=,tag:4JT50F72xL5Gyy5mqGjkrg==,type:str]
+sops:
+    kms: []
+    gcp_kms: []
+    azure_kv: []
+    hc_vault: []
+    age:
+        - recipient: age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw
+          enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBsR2ZvT21Bd2NuWnBXaFZa
+            OG9YUzg2b1g1VDQ3bHlaSGdMQW10UE9jYjEwCndSbWpDdis4S1IyOFdxL2U4eDVL
+            cG00RUxpcE1IRnR3cVk0dDJtK3Zwc00KLS0tIHhuNlVTakF1a3dyZFF4ZWg3TXFE
+            QzFjYmR1MVh3bVhydURmZ25ORmQxNGsKkcHUujLrlwycauIr1nJ9O9KG4T6NkmMv
+            8szwCF4AuqHzhSKhqGzYUnRQB1Igfpw85XjdR5O1aCf0mux/+E6Ihg==
+            -----END AGE ENCRYPTED FILE-----
+    lastmodified: "2021-04-27T04:12:50Z"
+    mac: ENC[AES256_GCM,data:darUiOLASNLXAMucmoQsaMvAk6KmMkb5owGfv38vsaF8JDXhxom0B6MTp2KRyiliv2KICEWv8BRofNXWQpgKGwvS8CPmZic7JgqdJhNpNVrt5NipisMcCigUVdk7WtQ73l7tTuqI5vBg9FnrnDgndnTr4CoqjxwThlbTtNy7+30=,iv:WmiLyZMMtsyL3KOTaki6EClKAyJ2KcVxanxcVJZGalo=,tag:ZJOBM+1NiyDN0a+OSndG6Q==,type:str]
+    pgp: []
+    unencrypted_regex: ^(kind|apiVersion|group|metadata)$
+    version: 3.7.1

examples/contrib/sops/gpg/README.md

@@ -0,0 +1,38 @@
+# SOPS PGP example
+
+SOPS function introduces for PGP case an additional `SOPS_IMPORT_PGP` ENV variable
+that must contain the private key(s) needed to decrypt yamls and public key(s) to
+encrypt yamls. If you have a file with keys it's possible to run:
+
+```sh
+kpt fn run --env SOPS_IMPORT_PGP="$(cat <file with exported key>.asc)" <folder>
+```
+
+or if your keys are already in stored in `gpg`-storage, it's possible to run:
+
+```sh
+kpt fn run --env SOPS_IMPORT_PGP="$(gpg --armor --export-secret-keys)" <folder>
+```
+
+## Function invocation
+
+Get this example and try it out by running the following commands:
+
+```sh
+# download sops kpt-function example
+kpt pkg get https://github.com/GoogleContainerTools/kpt-functions-catalog.git/examples/contrib/sops/gpg .
+
+# copy example GPG key from sops project
+curl -fsSL -o gpg_keys.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc
+
+# run the function to work with GPG
+kpt fn run --env SOPS_IMPORT_PGP="$(cat gpg_keys.asc)" gpg
+```
+
+## Expected result
+
+Verify the updated configuration:
+
+```sh
+kpt cfg cat gpg
+```

examples/contrib/sops/gpg/to-encrypt.yaml

@@ -0,0 +1,7 @@
+apiVersion: v1
+kind: somekind
+metadata:
+    name: toEncrypt
+nnn-password: password1
+user-password: password2
+k8s-password: password3

functions/contrib/ts/sops/build/sops.Dockerfile

@@ -2,7 +2,7 @@ FROM node:14.15-alpine3.12 as builder
 
 RUN apk add bash curl git && apk update
 
-ARG SOPS_VERSION="v3.6.1"
+ARG SOPS_VERSION="v3.7.1"
 RUN curl -fsSL -o /usr/local/bin/sops https://github.com/mozilla/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux && \
     chmod +x /usr/local/bin/sops
 
@@ -50,5 +50,6 @@ COPY --from=builder /usr/local/bin /usr/local/bin
 
 ENV PATH /usr/local/bin:$PATH
 ENV GNUPGHOME /tmp
+ENV XDG_CONFIG_HOME /tmp
 
 ENTRYPOINT ["node", "/home/node/app/dist/sops_run.js"]

functions/contrib/ts/sops/src/sops.ts

@@ -43,7 +43,9 @@ let cmdTolerateFailures = false;
 
 // pre-exec command may be overriden from config
 let preExecCmd =
-  '[ "$SOPS_IMPORT_PGP" == "" ] || (echo "$SOPS_IMPORT_PGP" | gpg --import)';
+  '[ "$SOPS_IMPORT_PGP" == "" ] || (echo "$SOPS_IMPORT_PGP" | gpg --import 2>/dev/null); \
+  [ "$XDG_CONFIG_HOME" == "" ] || [ "$SOPS_IMPORT_AGE" == "" ] || \
+  (mkdir -p $XDG_CONFIG_HOME/sops/age/ && echo "$SOPS_IMPORT_AGE" > $XDG_CONFIG_HOME/sops/age/keys.txt);';
 
 // list of annotations that will be detached before decryption
 // this is needed, because tools like kpt add some annotations

tests/sops.sh

@@ -29,13 +29,21 @@ kpt fn source example-configs |
   kpt fn run --env SOPS_IMPORT_PGP="$(cat key.asc)" --image gcr.io/kpt-fn-contrib/sops:"${TAG}" -- verbose=true >out.yaml
 assert_contains_string out.yaml "t00m4nys3cr3tzupdated"
 
-testcase "kpt_sops_declarative_example"
+testcase "kpt_sops_declarative_example_gpg"
 # get examples from the current version of repo
-cp -r "$REPODIR"/examples/contrib/sops .
+cp -r "$REPODIR"/examples/contrib/sops/gpg .
 curl -fsSL -o key.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc
-SOPS_IMPORT_PGP="$(cat key.asc)" kpt fn run sops
-assert_contains_string sops/to-decrypt.yaml "nnn-password: k8spassphrase"
-assert_contains_string sops/to-encrypt.yaml "nnn-password: 'ENC"
+SOPS_IMPORT_PGP="$(cat key.asc)" kpt fn run gpg
+assert_contains_string gpg/to-decrypt.yaml "nnn-password: k8spassphrase"
+assert_contains_string gpg/to-encrypt.yaml "nnn-password: 'ENC"
+
+testcase "kpt_sops_declarative_example_age"
+# get examples from the current version of repo
+cp -r "$REPODIR"/examples/contrib/sops/age .
+curl -fsSL -o keys.txt https://raw.githubusercontent.com/mozilla/sops/master/age/keys.txt
+SOPS_IMPORT_AGE="$(cat keys.txt)" kpt fn run age
+assert_contains_string age/to-decrypt.yaml "nnn-password: k8spassphrase"
+assert_contains_string age/to-encrypt.yaml "nnn-password: 'ENC"
 
 testcase "kpt_sops_declarative_fn_path"
 cat >fc.yaml <<EOF