DefaultFormatter.fillGeneral tends to panic if called outside a unit test context

[paul-at-cybr]

This line causes a panic if you run httpexpect outside a standard go unit test context, since the testing package hasn't been imported and its init-function hasn't run:

panic: testing: Verbose called before Parse

goroutine 1 [running]:
testing.Verbose(...)
        /usr/local/go/src/testing/testing.go:659
github.com/gavv/httpexpect/v2.(*DefaultFormatter).fillGeneral(0x0?, 0xc0000e3a40, 0xc0002e2930)
        /home/myuser/go/pkg/mod/github.com/gavv/httpexpect/[email protected]/formatter.go:322 +0x19d
github.com/gavv/httpexpect/v2.(*DefaultFormatter).buildFormatData(0xc0000d28f0?, 0xc0002e28c0?, 0xc00009edc0)
        /home/myuser/go/pkg/mod/github.com/gavv/httpexpect/[email protected]/formatter.go:259 +0x4a
github.com/gavv/httpexpect/v2.(*DefaultFormatter).applyTemplate(0x1?, {0x9e5d21, 0xf}, {0xa0571b, 0x589}, 0x40f088?, 0xc0002e2930?, 0x70?)
        /home/myuser/go/pkg/mod/github.com/gavv/httpexpect/[email protected]/formatter.go:237 +0x45
github.com/gavv/httpexpect/v2.(*DefaultFormatter).FormatFailure(0xc00031fae8?, 0x8aafa6?, 0x7f2f3604c108?)
        /home/myuser/go/pkg/mod/github.com/gavv/httpexpect/[email protected]/formatter.go:110 +0x7b
github.com/gavv/httpexpect/v2.(*DefaultAssertionHandler).Failure(0xc0000fef00, 0x1?, 0x1?)
        /home/myuser/go/pkg/mod/github.com/gavv/httpexpect/[email protected]/assertion.go:273 +0x4f
github.com/gavv/httpexpect/v2.(*chain).leave(0xc00031fb18?)
        /home/myuser/go/pkg/mod/github.com/gavv/httpexpect/[email protected]/chain.go:392 +0xbd
github.com/gavv/httpexpect/v2.(*Request).Expect(0xc0000e7040)
        /home/myuser/go/pkg/mod/github.com/gavv/httpexpect/[email protected]/request.go:1957 +0x175
github.com/my-org/my-project/tests.BfpDevTest(0xc00009e8c0?)
        /home/myuser/my-project/tests/bfpdev.go:42 +0x79
github.com/my-org/my-project/tester.(*Caddy).Run(0xc00009e8c0)
        /home/myuser/my-project/tester/caddy.go:64 +0x172
main.main()
        /home/myuser/my-project/cmd/test/main.go:20 +0x12f
exit status 2

The issue can be replicated by creating a custom logger that implements the httpexpect.TestingTB interface, and passing it to httpexpect.Default:

func NewHTTPExpect(name string, baseURL string) (*httpexpect.Expect, *Log) {
	tb := &Log{TestName: name}
	return httpexpect.Default(tb, baseURL), tb
}

The issue can be sidestepped by instead using httpexpect.WithConfig, and passing a httpexpect.DefaultFormatter that uses ColorModeAlways or ColorModeNever:

e := httpexpect.WithConfig(httpexpect.Config{
		TestName: name,
		BaseURL:  baseURL,
		Reporter: httpexpect.NewAssertReporter(tb),
		Formatter: &httpexpect.DefaultFormatter{
			ColorMode: httpexpect.ColorModeNever,
		},
	})

One way to fix this would be to check flag.Parsed() before calling testing.Verbose():

data.EnableColors = ctx.TestingTB && flag.Parsed() && testing.Verbose()

[paul-at-cybr]

Note that this issue probably can't be unit tested, since it relies on not running flag.Parse, which several functions in the testing package do.

[gavv]

Thanks for detailed report and PR! Merged.

BTW what is your use case that doesn't use testing? Just curious.

[paul-at-cybr]

I've found that go's unit test cache can be a hindrance when running e2e tests against backends that are not within the same repository as the test itself. The backend has changed, but the test will return cached results since the files it knows about remained unchanged.

I've concluded that the tests package is well optimized around assumptions that are specifically true for unit tests (and not e2e tests), and have decided to avoid the tests package altogether when doing e2e.

I might be entirely wrong about this approach 😄

[gavv]

Oh, I see. A silly question, do you know about -count=1 (official) hack? It will disable caching.

We can close the issue, right?

[paul-at-cybr]

Did not know about the -count=1 hack, that would probably have saved me some effort 😄
The issue can be closed, thanks for a very pleasant bug report experience!