e2e tests failing with no kind is registered for the type v1alpha1.Build

[apoorvajagtap]

Hello, we've been trying to perform e2e testing (without any changes yet). We have tried the following ways:

• Running GitHub Actions using act.
• Manually running the steps in same sequence as that in workflows/ci.yaml.

But, each time our e2e tests fails at random test-case, but mostly with the similar error of v1alpha1.Build not registered:

      /home/apoorva/projects/shipwright-io/build/test/e2e/e2e_test.go:641
    2023-06-27T07:53:10Z 1 Amended object: name='s2i-failing-xs2wv', image-url='ghcr.io/shipwright-io/build/build-e2e:s2i-failing'
    2023-06-27T07:53:10Z 1 Build s2i-failing-xs2wv created
    2023-06-27T07:54:13Z 1 Still waiting for build run 's2i-failing-xs2wv' to fail.
    [AfterEach] For a Kubernetes cluster with Tekton and build installed
      /home/apoorva/projects/shipwright-io/build/test/e2e/e2e_test.go:29
    2023-06-27T07:55:02Z 1 Print failed BuildRun's log
    2023-06-27T07:55:02Z 1 Failed to get Build from BuildRun s2i-failing-xs2wv: no kind is registered for the type v1alpha1.Build in scheme "k8s.io/client-go/kubernetes/scheme/register.go:74"
    2023-06-27T07:55:02Z 1 Failed to retrieve build and buildrun logs: no kind is registered for the type v1alpha1.Build in scheme "k8s.io/client-go/kubernetes/scheme/register.go:74"
    2023-06-27T07:55:02Z 1 The status of BuildRun s2i-failing-xs2wv: status=Unknown, reason=Pending 

We have tried these tests using the master branch as well as using v0.11.0 tag, but similar behavior.
We could also observe some stacktrace, however, this error seems to be primary. (Note: We could confirm that the v1alpha1 for build exists in the cluster.)
(tested against OpenShift as well as kind)

$ kubectl api-resources | grep -i build                                                                             [±tags/v0.11.0 ●]
buildruns                         br,brs                                 shipwright.io/v1alpha1                 true         BuildRun
builds                                                                   shipwright.io/v1alpha1                 true         Build
buildstrategies                   bs,bss                                 shipwright.io/v1alpha1                 true         BuildStrategy
clusterbuildstrategies            cbs,cbss                               shipwright.io/v1alpha1                 false        ClusterBuildStrategy

Any inputs/troubleshooting leads on the above will be really helpful. Thank you!

[jkhelil]

Pinging @qu1queee

[qu1queee]

To avoid back-and-forth, could we jump into a call tomorrow, so we can check directly in your cluster? @jkhelil @apoorvajagtap

[apoorvajagtap]

Hello, @qu1queee sure, I can help with the reproducing in my cluster. Anytime before 4:30 pm IST or after 6:30 pm IST tomorrow works for me.

[qu1queee]

We had a session today on this(@apoorvajagtap @qu1queee @HeavyWombat ). We reproduce the issue by running a single test case. We also ran with the kind cluster config from https://github.com/shipwright-io/build/blob/main/hack/install-kind.sh . After the test case failed, we modify the code to not clean up the resources and we manually try to simulate what the test case do, which is a GET operation of the Build CR. From the command line, the retrieval of the resource worked.

Next steps, @qu1queee to try to reproduce on his machine. @apoorvajagtap to rerun all test cases locally and see if all fail.

[apoorvajagtap]

Hello @qu1queee I tried running multiple test cases individually, but all of them were failing with similar errors.
I re-ran the whole test-suite, and looking through the summary [0], I think [1] could also be a concern.

Reading through the failure in [0], which basically tries to build an image using Buildah and a Git source , & then the GitError in [1] seems to be related (to me).
Please let me know your thoughts. Is there any additional configuration required to ensure that the build pods are able to access the git repo?

[0]

Summarizing 7 Failures:
  [FAIL] Using One-Off Builds Embed BuildSpec in BuildRun [It] should build an image using Buildah and a Git source
  /home/apoorva/projects/shipwright-io/build/test/e2e/validators_test.go:135
  [INTERRUPTED] For a Kubernetes cluster with Tekton and build installed when a Kaniko build with a Dockerfile that requires advanced permissions is defined [It] successfully runs a build
  /home/apoorva/projects/shipwright-io/build/test/e2e/e2e_test.go:415
 ...

[1]

$ oc get br buildah-hdh52 -ojson -n shipwright-build | jq .status.failureDetails                                            [±main ●]
{
  "location": {
    "container": "step-source-default",
    "pod": "buildah-hdh52-jw4f9-pod"
  },
  "message": "fatal:  unable to access 'https://github.com/shipwright-io/sample-go/': SSL certificate problem: self-signed certificate (exit code 128)",
  "reason": "GitError"
}