Allow to skip Cypress dashboard E2E tests using a label or env var (#…

…13146)

* Allow to switch Cypress command with cy2 on env var

* Remove coverage flag

* Restore Cypress dashboard

* Assign env var TEST_DISABLE_DASHBOARD_LABEL based on label presence

* Extend E2E script condition to TEST_DISABLE_DASHBOARD_LABEL

* Correct label term

* Remove unnecessary key and ID for Cypress command

* Correct PR label

* Add echo to display disable method; Add comments

* Add documentation

* Allow GH to set env var from the config

.github/workflows/test.yaml

@@ -22,6 +22,8 @@ env:
   TEST_BASE_URL: https://127.0.0.1/dashboard
   API: https://127.0.0.1
   TEST_PROJECT_ID: rancher-dashboard
+  TEST_DISABLE_DASHBOARD: ${{ vars.TEST_DISABLE_DASHBOARD }} # This is required to get it from the project configuration
+  TEST_DISABLE_DASHBOARD_LABEL: "${{ contains(github.event.pull_request.labels.*.name, 'ci/skip-e2e-cypress-dashboard')}}"
   CYPRESS_API_URL: http://139.59.134.103:1234/
   TEST_RUN_ID: ${{github.run_number}}-${{github.run_attempt}}-${{github.event.pull_request.title || github.event.head_commit.message}}
   # Build the dashboard to use in tests. When set to false it will grab `latest` from CDN (useful for running e2e tests quickly)

docusaurus/docs/internal/testing/e2e-test.md

@@ -109,6 +109,14 @@ These values are provided when you create a new project within Cypress dashboard
 
 It's also possible to run a workflow in GitHub Actions E2E test using these values to record on personal dashboards.
 
+### Skip dashboard or tests
+
+CI gates can be disabled in the following way:
+
+- Use label `ci/skip-e2e` to skip the E2E tests in the PR
+- Use label `ci/skip-e2e-cypress-dashboard` to run the E2E tests without Sorry Cypress dashboard in the PR (it will enable `TEST_DISABLE_DASHBOARD_LABEL` env var)
+- Use GitHub settings and define env var `TEST_DISABLE_DASHBOARD` as `true` (which is string and not boolean) to disable the Cypress dashboard entirely in every CI run
+
 ## Local and CI/prod run
 
 It is possible to start the project and run all the tests at once with a single command. There's however a difference between `dev` and `production` run. The first will not require an official certificate and will build the project in `dist`, while the production will enable all the SSL configurations to run encrypted.

package.json

@@ -44,7 +44,7 @@
     "e2e:dev": "START_SERVER_AND_TEST_INSECURE=1 server-test start:dev https-get://localhost:8005 cy:run:sorry",
     "e2e:build": "mkdir dist && TEST_INSTRUMENT=false ./scripts/build-e2e",
     "e2e:docker": "yarn docker:local:stop && ./scripts/e2e-docker-start $RANCHER_VERSION_E2E",
-    "e2e:prod": "BUILD_DASHBOARD=$BUILD_DASHBOARD GREP_TAGS=$GREP_TAGS TEST_USERNAME=$TEST_USERNAME TEST_BASE_URL=https://127.0.0.1/dashboard yarn cy:run",
+    "e2e:prod": "BUILD_DASHBOARD=$BUILD_DASHBOARD GREP_TAGS=$GREP_TAGS TEST_USERNAME=$TEST_USERNAME TEST_BASE_URL=https://127.0.0.1/dashboard yarn cy:run:sorry",
     "coverage": "npx nyc merge coverage coverage/coverage.json",
     "storybook": "cd storybook && yarn storybook",
     "build-storybook": "cd storybook && yarn install --no-lockfile && NODE_OPTIONS=--max_old_space_size=4096 yarn build-storybook --quiet",

scripts/e2e

@@ -10,4 +10,21 @@ if [ $# -eq 1 ]; then
   SPEC_ARG="--spec ${1}"
 fi
 
-CYPRESS_coverage=true CYPRESS_API_URL='http://139.59.134.103:1234' cy2 run ${SPEC_ARG} --group "$GREP_TAGS" --browser chrome --record --key rancher-dashboard --parallel --ci-build-id "$ID"
+# Define the command to use (cy2 or cypress)
+# TEST_DISABLE_DASHBOARD: Disable Cypress dashboard using GH repository settings
+# TEST_DISABLE_DASHBOARD_LABEL: Disable Cypress dashboard using PR label `ci/skip-e2e-cypress-dashboard`
+echo "TEST_DISABLE_DASHBOARD: ${TEST_DISABLE_DASHBOARD}"
+echo "TEST_DISABLE_DASHBOARD_LABEL: ${TEST_DISABLE_DASHBOARD_LABEL}"
+if [ "${TEST_DISABLE_DASHBOARD}" = "true" ] || [ "${TEST_DISABLE_DASHBOARD_LABEL}" = "true" ]; then
+  echo "Running Cypress without dashboard"
+  CYPRESS_COMMAND="cypress run ${SPEC_ARG}"
+else
+  echo "Running Cypress with Sorry Cypress on dashboard"
+  CYPRESS_COMMAND="cy2 run ${SPEC_ARG} --ci-build-id \"$ID\" --record --key rancher-dashboard --parallel --group \"$GREP_TAGS\""
+fi
+
+# Construct the full command
+E2E_COMMAND="CYPRESS_API_URL='http://139.59.134.103:1234' ${CYPRESS_COMMAND} --browser chrome"
+
+# Execute the command
+eval $E2E_COMMAND