contributing guide: add initial section on testing #1468

tpepper · 2017-12-06T19:27:56Z

This commit replaces the "Improvements needed" marker for the testing
topic with an initial section of overview information and jumping off links
to sig-testing, test documentation, and test code.

Part of: #1413

Signed-off-by: Tim Pepper [email protected]

tpepper · 2017-12-06T19:29:11Z

/assign @guineveresaenger @spiffxp

k8s-ci-robot · 2017-12-06T19:29:12Z

@tpepper: GitHub didn't allow me to assign the following users: guineveresaenger.

Note that only kubernetes members can be assigned.

In response to this:

/assign @guineveresaenger @spiffxp

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cblecker

It might also be worth noting in some place that CI runs these tests either as presubmits on PRs, post submits against master/release branches, or both (maybe even with a link to testgrid (https://testgrid.k8s.io)

cblecker · 2017-12-06T19:31:10Z

contributors/guide/README.md

-_Improvements needed_
-* link to testing process 
-* walkthrough of where to find what in the tests (how to use, how to debug)
+Testing is the responsibility of all contributors and is in part owned by all sigs, but is also coordinated by [sig-testing](https://github.com/kubernetes/community/tree/master/sig-testing).


Link should be: /sig-testing

cblecker · 2017-12-06T19:31:27Z

contributors/guide/README.md

-* walkthrough of where to find what in the tests (how to use, how to debug)
+Testing is the responsibility of all contributors and is in part owned by all sigs, but is also coordinated by [sig-testing](https://github.com/kubernetes/community/tree/master/sig-testing).
+
+The main testing overview document is [here](https://github.com/kubernetes/community/blob/master/contributors/devel/testing.md).


Link should be: /contributors/devel/testing.md

cblecker · 2017-12-06T19:31:58Z

contributors/guide/README.md

+
+The main testing overview document is [here](https://github.com/kubernetes/community/blob/master/contributors/devel/testing.md).
+
+There are three coarse types of test in kubernetes.  The location of the test code varies with type, as does the specifics of the environment needed to successfully run the test:


s/coarse/core/ ?

just dropped the word leaving "three types"

cblecker · 2017-12-06T19:32:39Z

contributors/guide/README.md

+
+There are three coarse types of test in kubernetes.  The location of the test code varies with type, as does the specifics of the environment needed to successfully run the test:
+
+* Unit: These confirm that a particular function behaves as intended.  Golang includes native ability for unit testing via the [testing](https://golang.org/pkg/testing/) package.  Unit test source code can be found adjacent to the corresponding source code within a given package.  For example: functions defined in [kubernetes/cmd/kubeadm/app/util/version.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version.go) will have unit tests in [kubernetes/cmd/kubeadm/app/util/version_test.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version_test.go).  These are easily run locally be any developer on any OS.


https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version.go should be https://git.k8s.io/kubernetes/cmd/kubeadm/app/util/version.go

https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version_test.go should be https://git.k8s.io/kubernetes/cmd/kubeadm/app/util/version_test.go

cblecker · 2017-12-06T19:33:42Z

contributors/guide/README.md

+There are three coarse types of test in kubernetes.  The location of the test code varies with type, as does the specifics of the environment needed to successfully run the test:
+
+* Unit: These confirm that a particular function behaves as intended.  Golang includes native ability for unit testing via the [testing](https://golang.org/pkg/testing/) package.  Unit test source code can be found adjacent to the corresponding source code within a given package.  For example: functions defined in [kubernetes/cmd/kubeadm/app/util/version.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version.go) will have unit tests in [kubernetes/cmd/kubeadm/app/util/version_test.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version_test.go).  These are easily run locally be any developer on any OS.
+* Integration: These tests cover interactions of package components or interactions between kubernetes components and some other non-kubernetes system resource (eg: etcd).  An example would be testing whether a piece of code can correctly store data to or retrieve data from etcd.  Integration tests are stored [here](https://github.com/kubernetes/kubernetes/tree/master/test/integration).  Running these can require the developer set up additional functionality on their development system.


https://github.com/kubernetes/kubernetes/tree/master/test/integration should be https://git.k8s.io/kubernetes/test/integration

cblecker · 2017-12-06T19:34:09Z

contributors/guide/README.md

+
+* Unit: These confirm that a particular function behaves as intended.  Golang includes native ability for unit testing via the [testing](https://golang.org/pkg/testing/) package.  Unit test source code can be found adjacent to the corresponding source code within a given package.  For example: functions defined in [kubernetes/cmd/kubeadm/app/util/version.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version.go) will have unit tests in [kubernetes/cmd/kubeadm/app/util/version_test.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version_test.go).  These are easily run locally be any developer on any OS.
+* Integration: These tests cover interactions of package components or interactions between kubernetes components and some other non-kubernetes system resource (eg: etcd).  An example would be testing whether a piece of code can correctly store data to or retrieve data from etcd.  Integration tests are stored [here](https://github.com/kubernetes/kubernetes/tree/master/test/integration).  Running these can require the developer set up additional functionality on their development system.
+* End-to-end ("e2e"): These are broad tests of overall kubernetes system behavior and coherence.  These are more complicated as they require a functional kubernetes cluster built from the sources to be tested.  A separate document [here](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md) details e2e testing and test cases themselves can be found [here](https://github.com/kubernetes/kubernetes/tree/master/test/e2e).


https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md should be /contributors/devel/e2e-tests.md

https://github.com/kubernetes/kubernetes/tree/master/test/e2e should be https://git.k8s.io/kubernetes/test/e2e

cblecker · 2017-12-06T19:35:05Z

contributors/guide/README.md

+* Integration: These tests cover interactions of package components or interactions between kubernetes components and some other non-kubernetes system resource (eg: etcd).  An example would be testing whether a piece of code can correctly store data to or retrieve data from etcd.  Integration tests are stored [here](https://github.com/kubernetes/kubernetes/tree/master/test/integration).  Running these can require the developer set up additional functionality on their development system.
+* End-to-end ("e2e"): These are broad tests of overall kubernetes system behavior and coherence.  These are more complicated as they require a functional kubernetes cluster built from the sources to be tested.  A separate document [here](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md) details e2e testing and test cases themselves can be found [here](https://github.com/kubernetes/kubernetes/tree/master/test/e2e).
+
+Sig-testing is responsible for official infrastructure and its associated automation, which is tracked in the [test-insfra repo](https://github.com/kubernetes/test-infra).  If you're looking to run e2e tests on your own infrastructure, [kubetest](https://github.com/kubernetes/test-infra/tree/master/kubetest) is the mechanism.


s/Sig-testing/sig-testing/
s/test-insfra/test-infra/

https://github.com/kubernetes/test-infra should be https://git.k8s.io/test-infra

https://github.com/kubernetes/test-infra/tree/master/kubetest should be https://git.k8s.io/test-infra/kubetest

tpepper · 2017-12-06T23:21:46Z

I think I hit all @cblecker 's comments and added the suggested CI & results blurb.

I thought about also adding a blurb on flakes, especially as a newbie contributor's first touch simple PR's are highly likely to hit job flakes that are clearly unrelated to their PR's (eg: simple comment or link url or typo or otherwise super trivial fixups). The section's already getting long though and @guineveresaenger doesn't want this doc to be large wall of text, so maybe that's better off in a testing FAQ elsewhere.

cblecker · 2017-12-07T00:48:01Z

contributors/guide/README.md

+
+There are three types of test in kubernetes.  The location of the test code varies with type, as does the specifics of the environment needed to successfully run the test:
+
+* Unit: These confirm that a particular function behaves as intended.  Golang includes native ability for unit testing via the [testing](https://golang.org/pkg/testing/) package.  Unit test source code can be found adjacent to the corresponding source code within a given package.  For example: functions defined in [kubernetes/cmd/kubeadm/app/util/version.go](https://git.k8s.io/kubernetes/cmd/kubeadm/app/util/version.go) will have unit tests in [kubernetes/cmd/kubeadm/app/util/version_test.go](https://git.k8s.io/kubernetes/cmd/kubeadm/app/util/version.go).  These are easily run locally be any developer on any OS.


the version_test.go link leads to version.go

Dang copy paste while sitting in a conference talk multitasking. Refixed.

cblecker · 2017-12-07T00:48:54Z

lgtm other than the one fix above. will leave for others to have a chance to look at
/ok-to-test
/approve

This commit replaces the "Improvements needed" marker for the testing topic with an initial section of overview information and jumping off links to sig-testing, test documentation, and test code. Part of: kubernetes#1413 Signed-off-by: Tim Pepper <[email protected]>

guineveresaenger · 2017-12-07T18:29:53Z

@tpepper this is beautiful! Thank you so much. This is concise, clear, and it helped me pull some bits of knowledge together to understand more, just by reading this PR.

@spiffxp someone with the power should merge this!

cblecker · 2017-12-08T17:37:01Z

/lgtm

k8s-ci-robot · 2017-12-08T17:37:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cblecker, tpepper

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~OWNERS~~ [cblecker]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

k8s-github-robot · 2017-12-08T17:43:30Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2017-12-08T17:45:04Z

Automatic merge from submit-queue.

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 6, 2017

k8s-ci-robot assigned spiffxp Dec 6, 2017

cblecker requested changes Dec 6, 2017

View reviewed changes

tpepper force-pushed the contrib_guide branch from 1ec9d24 to 81c7ee3 Compare December 6, 2017 23:17

cblecker reviewed Dec 7, 2017

View reviewed changes

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 7, 2017

tpepper force-pushed the contrib_guide branch from 81c7ee3 to 951c10e Compare December 7, 2017 03:27

k8s-ci-robot assigned cblecker Dec 8, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 8, 2017

cblecker approved these changes Dec 8, 2017

View reviewed changes

k8s-github-robot merged commit 1d05904 into kubernetes:master Dec 8, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

contributing guide: add initial section on testing #1468

contributing guide: add initial section on testing #1468

tpepper commented Dec 6, 2017

tpepper commented Dec 6, 2017

k8s-ci-robot commented Dec 6, 2017

cblecker left a comment

cblecker Dec 6, 2017

tpepper Dec 6, 2017

cblecker Dec 6, 2017

tpepper Dec 6, 2017

cblecker Dec 6, 2017

tpepper Dec 6, 2017

cblecker Dec 6, 2017

cblecker Dec 6, 2017

tpepper Dec 6, 2017

cblecker Dec 6, 2017

tpepper Dec 6, 2017

cblecker Dec 6, 2017

cblecker Dec 6, 2017

tpepper Dec 6, 2017

cblecker Dec 6, 2017

cblecker Dec 6, 2017

cblecker Dec 6, 2017

tpepper Dec 6, 2017

tpepper commented Dec 6, 2017

cblecker Dec 7, 2017

tpepper Dec 7, 2017

cblecker commented Dec 7, 2017

guineveresaenger commented Dec 7, 2017

cblecker commented Dec 8, 2017

k8s-ci-robot commented Dec 8, 2017

k8s-github-robot commented Dec 8, 2017

k8s-github-robot commented Dec 8, 2017


		The main testing overview document is [here](https://github.com/kubernetes/community/blob/master/contributors/devel/testing.md).

		There are three coarse types of test in kubernetes. The location of the test code varies with type, as does the specifics of the environment needed to successfully run the test:


		There are three coarse types of test in kubernetes. The location of the test code varies with type, as does the specifics of the environment needed to successfully run the test:

		* Unit: These confirm that a particular function behaves as intended. Golang includes native ability for unit testing via the [testing](https://golang.org/pkg/testing/) package. Unit test source code can be found adjacent to the corresponding source code within a given package. For example: functions defined in [kubernetes/cmd/kubeadm/app/util/version.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version.go) will have unit tests in [kubernetes/cmd/kubeadm/app/util/version_test.go](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/version_test.go). These are easily run locally be any developer on any OS.


		There are three types of test in kubernetes. The location of the test code varies with type, as does the specifics of the environment needed to successfully run the test:

		* Unit: These confirm that a particular function behaves as intended. Golang includes native ability for unit testing via the [testing](https://golang.org/pkg/testing/) package. Unit test source code can be found adjacent to the corresponding source code within a given package. For example: functions defined in [kubernetes/cmd/kubeadm/app/util/version.go](https://git.k8s.io/kubernetes/cmd/kubeadm/app/util/version.go) will have unit tests in [kubernetes/cmd/kubeadm/app/util/version_test.go](https://git.k8s.io/kubernetes/cmd/kubeadm/app/util/version.go). These are easily run locally be any developer on any OS.

contributing guide: add initial section on testing #1468

contributing guide: add initial section on testing #1468

Conversation

tpepper commented Dec 6, 2017

tpepper commented Dec 6, 2017

k8s-ci-robot commented Dec 6, 2017

cblecker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tpepper commented Dec 6, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cblecker commented Dec 7, 2017

guineveresaenger commented Dec 7, 2017

cblecker commented Dec 8, 2017

k8s-ci-robot commented Dec 8, 2017

k8s-github-robot commented Dec 8, 2017

k8s-github-robot commented Dec 8, 2017