Required Values in templates not being tested cause cryptic error messages

[AaronKalair]

Hi,

Really like this project, and it was good to see Helm 3 support being added!

I was stumped by some cryptic error messages I was getting from Helm Unittest today:

	- manifest should match snapshot

		- asserts[0] `matchSnapshot` fail

			Error:
				template "debug/templates/b.yaml" not exists or not selected in test suite

I've managed to workout what causes it though, if you have required values inside of a Helm Template (Using the required function e.g. {{ required ".someParameter is required" $.Values.someParameter }}.

Then even if you write a test for a different template that doesn't need this value, you get the above error.

To replicate:

helm create debug 
cd debug
rm templates/*
rm -rf templates/tests

Create a.yaml in the templates directory

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ required ".someParameter is required" $.Values.someParameter }}
  namespace: "test"
data:
    a: "a"

Create b.yaml in the templates directory

apiVersion: v1
kind: ConfigMap
metadata:
  name: IDontNeedParameters
  namespace: "iWork"
data:
    b: "b"

Create a test case in tests/b_test.yaml

templates:
  - b.yaml
tests:
  - it: manifest should match snapshot
    asserts:
      - matchSnapshot: {}

Notice that this solely tests b.yaml that has no required values

Run Helm Unit Test

helm unittest -3 .

You'll get this error:

### Chart [ debug ] .

 FAIL  	tests/b_test.yaml
	- manifest should match snapshot

		- asserts[0] `matchSnapshot` fail

			Error:
				template "debug/templates/b.yaml" not exists or not selected in test suite


Charts:      1 failed, 0 passed, 1 total
Test Suites: 1 failed, 0 passed, 1 total
Tests:       1 failed, 0 passed, 1 total
Snapshot:    0 passed, 0 total
Time:        9.265483ms

Set someParameter in the test:

templates:
  - b.yaml
tests:
  - it: manifest should match snapshot
    set:
      someParameter: "fake"
    asserts:
      - matchSnapshot: {}

Run Helm Unit test again

helm unittest -3 .

The tests now pass:

### Chart [ debug ] .

 PASS  	tests/b_test.yaml

Charts:      1 passed, 1 total
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshot:    1 passed, 1 total
Time:        2.310412ms

[quintush]

Hello @AaronKalair,

Thanks for the feedback and the clear example.
I will look into this.

Greetings,
@quintush

[AaronKalair]

Thanks for fixing this!

[0xStarcat]

I have a similar setup & example using helm's required function, however i'm using the failedTemplate assertion and get this cryptic message: yaml: found character that cannot start any token

When providing a value it goes away.