Add helpers for basic acceptance test

[jrhouston]

Description

Closes #7

This PR adds a very basic acceptance test and brings in the terraform-plugin-test helper to drive the tests using the terraform binary.

I have left the test deliberately indirect at the moment to open discussion. We are likely to re-invent parts of the existing test framework because we can't drop them in here. Something to consider is making it easy for ourselves to refactor to the new SDK when it becomes available, without inconveniencing ourselves too much while we develop this in the meantime.

For convenience I have introduced functions for:

• Creating and destroying namespaces
• Asserting that resources exist / do not exist
• Picking out the object attribute and asserting that fields have a particular value

Run the tests using make testacc as usual.

References

https://github.com/hashicorp/terraform-plugin-test

Unblocks #15

[jrhouston]

This was the very simplest test I could think of.

[alexsomesan]

Sweet! I tried it out quickly and works nicely.
Dropped some conversation starters here and there and I'll play more with it in the meantime.

[jrhouston]

I've tidied the helper functions into a couple of helper packages, one for kubernetes and another for making assertions against the tfstate so it's easy to write statements like:

tfstate.AssertAttributeEqual(t, "kubernetes_manifest.test.object.data.foo", "bar")

I don't really want to take it any further than this as the test is pretty legible, and this is pretty much all we need for the moment. We can add in more things as we go and this rough format will be easy to port over to the SDK style of tests when the time comes to do that.

[jrhouston]

Something else I've done here is put the terraform configs for the tests into their own files in the testdata directory.

It always bothered me that the configs in other providers were full of go formatting directives. You couldn't just grab a test config and run it by hand with terraform without editing out the formatting directives.

As an alternative, I'm using terraform's own variable syntax here and setting them when we load the config file.

[alexsomesan]

Very cool! Totally agree with having them in stand alone files for the exact same reason. I always wanted to be able to easily run a test config by hand when I need to.

On Sat, May 9, 2020 at 7:26 PM John Houston ***@***.***> wrote: Something else I've done here is put the terraform configs for the tests into their own files in the testdata directory. It always bothered me that the configs in other providers were full of go formatting directives. You couldn't just grab a test config and run it by hand with terraform without editing out the formatting directives. As an alternative, I'm using terraform's own variable syntax here and setting them when we load the config file. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIL5G535PP7X4HTQQAQ52LRQWG3JANCNFSM4MZ67ARQ> .

-- — Sent from my phone.

[jrhouston]

I have a PR open to address this here hashicorp/terraform-plugin-test#24

[jrhouston]

Have added a -server-side-plan flag so we can run the suite twice with it enabled and disabled, rather than having multiple explicit tests.

[alexsomesan]

LGTM. Cool stuff, @jrhouston !

[alexsomesan]

For when this might grow more complex, we should keep in mind that Golang has a dedicated text templating mechanism that nicely separates the "model" from the "view", to make an old-fashioned analogy :)
It's detailed here: https://golang.org/pkg/text/template

Just food for thought, no need to change in this now.

[jrhouston]

That's true - but if we use template then we're back to having edit out go template directives before we can run the acceptance test fixtures by hand. I'm hoping we can do everything just by supplying input variables and make use of HCL's var, and for_each and so on.

[alexsomesan]

That makes sense. I agree, the convenience running the test configs by hand should always be a consideration.

On Mon, May 25, 2020 at 8:09 PM John Houston ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In test/acceptance/acceptance_test.go <#20 (comment)> : > + _, currentFilename, _, ok := runtime.Caller(0) + if !ok { + panic("Could not determine testdir directory") + } + + testdata := path.Dir(currentFilename) + tfconfig, err := ioutil.ReadFile(fmt.Sprintf("%s/testdata/%s", testdata, filename)) + if err != nil { + t.Fatal(err) + return "" + } + + // FIXME HACK this is something we could probably add to the binary test helper + // and it can supply the -var flag instead of doing this + vars := "" + for name, value := range tfvars { That's true - but if we use template then we're back to having edit out go template directives before we can run the acceptance test fixtures by hand. I'm hoping we can do everything just by supplying input variables and make use of HCL's var, and for_each and so on. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIL5G4LCFV4TZRFM4KHXOTRTKX55ANCNFSM4MZ67ARQ> .

-- — Sent from my phone.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!