Skip to content
This repository has been archived by the owner on Jun 28, 2023. It is now read-only.

Adding shell scripts tests in e2e-framework #2246

Merged
merged 1 commit into from
Nov 3, 2021

Conversation

aman556
Copy link
Contributor

@aman556 aman556 commented Oct 13, 2021

What this PR does / why we need it

This PR have two motive in e2e-framework.

  • First it will add shell scripts to test :
    • Fluent-bit
    • cert-manager
    • contour
    • local-path-storage

Details for the Release Notes (PLEASE PROVIDE)

e2e-framework can test the packages like contour, fluent-bit, cert-manager and local-path-storage. No change in the
commands in e2e-framework for testing these packages. 
e2e-framework will be also add as a plugin in github workflow.

Which issue(s) this PR fixes

Fixes: #2243

Describe testing done for PR

Run ginkgo -v -- --packages="" --version="" --guest-cluster-name="" to check the working of shell scripts.

Special notes for your reviewer

The bash scripts to test packages follow these steps:

  • Running tanzu package installed list to check the package which we are testing is already installed or not.
  • Running tanzu package available list contour.community.tanzu.vmware.com for fetching the latest version of that package.
  • Running tanzu package install pkgName --package-name pkgName.community.tanzu.vmware.com -f config.yaml --version "${version}" to install the package.
  • Taking care of all the prerequisites.
  • Ensuring that the package is working fine.
  • Deleting all the prerequisites and the package.

@aman556 aman556 requested review from a team as code owners October 13, 2021 18:28
@github-actions github-actions bot added the owner/release-eng Work executed by VMware release engineering team label Oct 13, 2021
@jpmcb jpmcb changed the title [WIP] Adding shell scripts tests in e2e-framework Adding shell scripts tests in e2e-framework Oct 13, 2021
@jpmcb jpmcb marked this pull request as draft October 13, 2021 18:43
@jpmcb
Copy link
Contributor

jpmcb commented Oct 13, 2021

FYI - converting this to a "Draft" PR. Please feel free to mark as ready when it's good for reviews

@github-actions github-actions bot added the owner/packages Work executed by a package's maintainer label Oct 19, 2021
@github-actions github-actions bot removed the owner/packages Work executed by a package's maintainer label Oct 29, 2021
@aman556
Copy link
Contributor Author

aman556 commented Oct 29, 2021

Latest approach:
The bash scripts are working as a smoke tests which are kept in test/e2e path with the structure.

├── Makefile
└── packages
    └── cert-manager
        └── 1.5.3
            └── cert-manager-test.sh
  • The Makefile have make targets for each package to run the smoke tests as well as the ginkgo tests for the folders which have these test. All the changes are in test/e2e/smoke.
  • make smoke-test will run all the ginkgo and smoke tests.
  • To understand what shell scripts are doing i have explained in the special notes. There are smoke tests for other packages too will add in follow up PR's.
  • In future using test/e2e/smoke-test/Makefile we will try to add this in github workflow.

@karuppiah7890
Copy link
Contributor

@karuppiah7890
Copy link
Contributor

karuppiah7890 commented Oct 29, 2021

@aman556 If we plan to add GitHub workflows later in a separate PR, I would recommend updating the PR description including the release notes to reflect the expected state of this PR so that reviewing is easy

@aman556
Copy link
Contributor Author

aman556 commented Nov 1, 2021

Thanks @karuppiah7890 for the feedback. I have fixed the golangci-lint error and also updated the PR description.

Makefile Outdated Show resolved Hide resolved
hack/tagger/go.mod Outdated Show resolved Hide resolved
@karuppiah7890
Copy link
Contributor

TODO

  • Ensure there are no changes in Makefile file
  • Ensure there are no changes in hack/tagger directory

@karuppiah7890
Copy link
Contributor

General observation - we can rethink our sleep calls. I can see that it works, I just tried it out, but at times it may not work - as in, it may not be robust - sometimes we might need more sleep time. And adding more and more time to sleep can make the test run longer and also have a constant sleep / wait time. We can instead try to use kubectl wait wherever possible to have a non-constant sleep time but still maintain a max time out since we cannot sleep / wait indefinitely

@karuppiah7890
Copy link
Contributor

Let's take care of improving and optimizing the sleep / wait strategy in a separate PR

hack/tagger/go.mod Outdated Show resolved Hide resolved
@karuppiah7890 karuppiah7890 marked this pull request as ready for review November 3, 2021 11:57
@karuppiah7890 karuppiah7890 force-pushed the e2e-framework branch 2 times, most recently from ea61ff0 to 5821194 Compare November 3, 2021 12:18
Why? As some packages do not have testing scripts.
Shell scripts in this commit will install package
and prerequisites and check it's working and then
do the clean up

Signed-off-by: Aman Sharma <[email protected]>
Signed-off-by: Karuppiah Natarajan <[email protected]>
Copy link
Contributor

@karuppiah7890 karuppiah7890 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! There are still some small nitty gritty things to take care of, which we can take care separately and also some things to smoothen out the rough edges. To avoid longer PR review times and longer PR changes, I'm going to merge this off. Let's pair on the next PR @aman556 and work on the e2e framework including the #2167 PR

I have squashed all the commits and put a small commit message (mostly from the first commit message of all the commits)

Merging it now!

@karuppiah7890 karuppiah7890 merged commit a34aac7 into vmware-tanzu:main Nov 3, 2021
@aman556
Copy link
Contributor Author

aman556 commented Nov 3, 2021

Thanks @karuppiah7890 for giving your time in reviewing this PR and merging it will improve all the above things in the follow-up PR. Agree with all the changes for the improvement of the script. Thanks again Karuppiah means allot.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-not-required owner/release-eng Work executed by VMware release engineering team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

e2e-framework can not test all the Packages
4 participants