test: Cypress | Helpers improved + Flaky fixes #30735

Aishwarya-U-R · 2024-01-30T06:57:28Z

Description

This PR fixes the RenameApplication flakyness in CI with added dynamic check
Also replacing js cy.renameApp to TS helper
Flaky fixes - cypress/e2e/Regression/ClientSide/Workspace/MemberRoles_Spec.ts (entire spec updates for EnableGAC, removed signout from 'it' blocks)
cypress/e2e/Regression/ClientSide/Workspace/ShareAppTests_Spec.ts (7th flaky fixed)
cypress/e2e/Regression/ClientSide/SetProperty/WidgetPropertySetters2_spec.ts (5th test)
cypress/e2e/Regression/ClientSide/Templates/Fork_Template_Existing_app_spec.js (2nd - added validation to match the test description, 1st & 3rd - removed static waits, Added multiple dynamic checks)
cypress/e2e/Regression/ClientSide/OtherUIFeatures/ApplicationURL_spec.js (3rd & 4th flaky tests)
homePage.AssertViewPageLoad() created
homePage.LaunchAppFromAppHover() improved
homePage.Signout() - added dynamic checks
Added more validation to homePage.Signup() method with Dynamic checks
homePage.LeaveWorkspace() removed redundant SelectWorkspace call
admingSettings.EnableGAC() - added dynamic checks
featureFlagIntercept - removed static sleep, reload check improved
agHelper.VisitNAssert() - removed static sleep
homePage.OpenMembersPageForWorkspace() - removed sleep, added dynamic checks

Type of change

Script fix (non-breaking change which fixes an issue)

Testing

How Has This Been Tested?

Cypress CI runs

Checklist:

QA activity:

Added Test Plan Approved label after Cypress tests were reviewed

Summary by CodeRabbit

Refactor
- Enhanced Cypress test commands across multiple test suites for improved efficiency and readability.
- Refactored conditional checks and method invocations for better test scenario handling.
Tests
- Updated testing approaches for application deployment, workspace management, and error handling.
- Introduced new assertions for UI visibility and functional behavior in automated tests.
Chores
- Optimized GitHub Actions workflow by adjusting the matrix count for build processes.
- Added new test specs for limited tests in the client-side regression suite.

coderabbitai · 2024-01-30T06:58:20Z

Walkthrough

This set of changes across various Cypress test files and support structures in the codebase primarily focuses on enhancing test reliability and maintainability. By leveraging helper functions and classes for UI interactions, assertions, and setup tasks, the modifications aim to streamline the testing process. Furthermore, adjustments in feature flag handling and workflow optimizations reflect an effort to improve test execution efficiency and adapt to application changes, ensuring that tests are both robust and aligned with current development standards.

Changes

Files	Summary of Changes
`ApplicationURL_spec.js`, `MemberRoles_Spec.ts`, `ShareAppTests_Spec.ts`, `WidgetPropertySetters2_spec.ts`	Refactored UI interactions and assertions using helper functions; adjusted logic based on repository conditions.
`AdminSettings.ts`, `HomePage.ts`	Enhanced assertions for UI elements; modified action sequences and added visibility checks.
`Migration_Spec.js`, `WorkspaceCommands.js`, `gitSync.js`	Streamlined application renaming and setup processes using helper functions; removed unnecessary wait calls.
`FeatureFlags.ts`, `commands.js`	Refactored feature flag handling and API response modification; improved test stability with dynamic data handling.
`Fork_Template_Existing_app_spec.js`, `AggregateHelper.ts`	Optimized control flow by adjusting assertions and commenting out redundant wait calls.
`.github/workflows/build-client-server.yml`	Adjusted workflow configuration for more efficient execution.

Related issues

[Task]: Fix cypress test suite failures caused by PR #29794 #30378: The refactoring and enhancements in test reliability and maintainability potentially address underlying causes of test failures mentioned in this issue. The modifications aim to ensure that the test suite runs successfully, which aligns with the objective of rectifying specific test case failures due to prior changes.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit-tests for this file.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit tests for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository from git and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit tests.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
The JSON schema for the configuration file is available here.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

github-actions · 2024-01-30T07:00:30Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7707439413.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-30T07:23:14Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7707439413.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-01-30T07:27:16Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7707692566.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-30T08:17:29Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7707692566.
Cypress dashboard: Click here!
The following are new failures, please fix them before merging the PR:

cypress/e2e/Regression/ClientSide/Git/ExistingApps/v1.9.24/DSCrudAndBindings_Spec.ts
cypress/e2e/Regression/ClientSide/Git/GitSync/GitSyncedApps_spec.js
cypress/e2e/Regression/ClientSide/IDE/MaintainContext&Focus_spec.js
cypress/e2e/Regression/ClientSide/Onboarding/FirstTimeUserOnboarding_spec.js
cypress/e2e/Regression/ClientSide/PartialImportExport/PartialExport_spec.ts
cypress/e2e/Regression/ClientSide/Templates/ForkTemplateToGitConnectedApp.js
cypress/e2e/Regression/ClientSide/Widgets/Multiselect/MultiSelect5_spec.ts
cypress/e2e/Regression/ClientSide/Workspace/MemberRoles_Spec.ts
cypress/e2e/Regression/ServerSide/QueryPane/S3_1_spec.js

To know the list of identified flaky tests - Refer here

Aishwarya-U-R · 2024-01-30T11:04:54Z

/ci-test-limit runId=7707692566

Aishwarya-U-R · 2024-01-30T11:05:17Z

/ci-test-limit runId=7707439413

github-actions · 2024-01-30T11:05:30Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7710157073.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-30T11:06:26Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7710167207.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-30T11:07:25Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7710157073.
Cypress dashboard: Click here!
The following are new failures, please fix them before merging the PR:

To know the list of identified flaky tests - Refer here

github-actions · 2024-01-30T11:07:36Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7710167207.
Cypress dashboard: Click here!
The following are new failures, please fix them before merging the PR:

To know the list of identified flaky tests - Refer here

Aishwarya-U-R · 2024-01-30T11:10:07Z

/ci-test-limit

github-actions · 2024-01-30T11:10:37Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7710226276.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-30T11:48:06Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7710226276.
Cypress dashboard: Click here!
The following are new failures, please fix them before merging the PR:

cypress/e2e/Regression/ClientSide/Workspace/MemberRoles_Spec.ts

To know the list of identified flaky tests - Refer here

Aishwarya-U-R · 2024-01-30T11:50:52Z

/ci-test-limit runId=7710226276

github-actions · 2024-01-30T11:51:22Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7710667835.
Workflow: ci-test-limited.
PR: 30735.

Aishwarya-U-R · 2024-01-30T11:57:22Z

/ci-test-limit runId=7710226276

github-actions · 2024-01-30T11:57:54Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7710731500.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-31T18:14:45Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7730547320.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-31T18:22:49Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7730539007.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-01-31T18:22:57Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7730536874.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-01-31T18:23:44Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7730547320.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-01-31T18:31:47Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7730737360.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-01-31T19:24:48Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7730737360.
Cypress dashboard: Click here!
The following are new failures, please fix them before merging the PR:

cypress/e2e/Regression/ClientSide/Anvil/AnvilAppNavigation_spec.ts
cypress/e2e/Regression/ClientSide/Anvil/AnvilDnD_spec.ts
cypress/e2e/Regression/ClientSide/Git/ExistingApps/v1.9.24/DSCrudAndBindings_Spec.ts
cypress/e2e/Regression/ClientSide/Git/GitSync/GitSyncedApps_spec.js
cypress/e2e/Regression/ClientSide/IDE/MaintainContext&Focus_spec.js
cypress/e2e/Regression/ClientSide/Onboarding/FirstTimeUserOnboarding_spec.js
cypress/e2e/Regression/ClientSide/PartialImportExport/PartialExport_spec.ts
cypress/e2e/Regression/ClientSide/Templates/ForkTemplateToGitConnectedApp.js
cypress/e2e/Regression/ClientSide/Templates/Fork_Template_To_App_spec.js
cypress/e2e/Regression/ClientSide/Widgets/Multiselect/MultiSelect5_spec.ts
cypress/e2e/Regression/ClientSide/Widgets/Select/RTL_support.ts

To know the list of identified flaky tests - Refer here

Aishwarya-U-R · 2024-02-01T07:14:55Z

/ok-to-test tags="@tag.All"

github-actions · 2024-02-01T07:15:17Z

Whoa, @tag.All spotted in your test suite! 🚀
While @tag.All is cool, like a catch-all net, why not try specific tags? 🏷️
Narrow down your suite with specific tags for quicker and more accurate tests! 🚀 Less waiting, more zipping through tests like a ninja!
Explore the tags documentation here

github-actions · 2024-02-01T07:15:18Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7737593997.
Workflow: Appsmith External Integration Test Workflow.
Tags: ``.

github-actions · 2024-02-01T08:05:58Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7737593997.
Commit: ``.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-02-01T08:10:04Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7738113489.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-02-01T08:10:22Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7738116847.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-02-01T08:19:37Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7738116847.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-02-01T08:19:37Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7738113489.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-02-01T08:21:08Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7738235633.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-02-01T08:21:20Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/7738238351.
Workflow: ci-test-limited.
PR: 30735.

github-actions · 2024-02-01T08:30:14Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7738235633.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

github-actions · 2024-02-01T08:30:16Z

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/7738238351.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

This reverts commit 9fe9af1.

RenameApplication & Signup improved

91731a8

github-actions bot added skip-changelog Adding this label to a PR prevents it from being listed in the changelog Tests Test issues labels Jan 30, 2024

Merge branch 'release' into test/renameappflakyfix

6b37139

run all

c5cd8f6

Aishwarya-U-R changed the title ~~test: Cypress | Helper methods RenameApplication & Signup() improved with Dynamic checks~~ test: Cypress | Helpers RenameApplication & Signup() improved with Dynamic checks Jan 30, 2024

Aishwarya-U-R changed the title ~~test: Cypress | Helpers RenameApplication & Signup() improved with Dynamic checks~~ test: Cypress | Helpers RenameApplication() & Signup() improved with Dynamic checks Jan 30, 2024

Aishwarya-U-R changed the title ~~test: Cypress | Helpers RenameApplication() & Signup() improved with Dynamic checks~~ test: Cypress | Helpers RenameApplication() & Signup() improved Jan 30, 2024

MemberRoles_Spec fix

a751d91

Merge branch 'release' into test/renameappflakyfix

8730712

fix

3365f71

fix

e2b75fb

Aishwarya-U-R added 2 commits January 31, 2024 23:59

run all

d22c7f0

Merge branch 'release' into test/renameappflakyfix

00b6048

Aishwarya-U-R changed the title ~~test: Cypress | Helpers update + Flaky fixes~~ test: Cypress | Helpers improved + Flaky fixes Jan 31, 2024

remove comments

0ad2199

Merge branch 'release' into test/renameappflakyfix

8169919

revert

6c44ef8

Aishwarya-U-R requested a review from sarojsarab February 1, 2024 08:06

running fixed tests

9fe9af1

Revert "running fixed tests"

0e2f658

This reverts commit 9fe9af1.

sarojsarab approved these changes Feb 1, 2024

View reviewed changes

Aishwarya-U-R merged commit 67d20d9 into release Feb 1, 2024
16 checks passed

Aishwarya-U-R deleted the test/renameappflakyfix branch February 1, 2024 11:12

test: Cypress | Helpers improved + Flaky fixes #30735

test: Cypress | Helpers improved + Flaky fixes #30735

Conversation

Aishwarya-U-R commented Jan 30, 2024 • edited Loading

Description

Type of change

Testing

How Has This Been Tested?

Checklist:

QA activity:

Summary by CodeRabbit

Summary by CodeRabbit

coderabbitai bot commented Jan 30, 2024 • edited Loading

Walkthrough

Changes

Related issues

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

CodeRabbit Discord Community

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

Aishwarya-U-R commented Jan 30, 2024

Aishwarya-U-R commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

Aishwarya-U-R commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

Aishwarya-U-R commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

Aishwarya-U-R commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 31, 2024

github-actions bot commented Jan 31, 2024

github-actions bot commented Jan 31, 2024

github-actions bot commented Jan 31, 2024

github-actions bot commented Jan 31, 2024

github-actions bot commented Jan 31, 2024

Aishwarya-U-R commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 1, 2024

Aishwarya-U-R commented Jan 30, 2024 •

edited

Loading

coderabbitai bot commented Jan 30, 2024 •

edited

Loading

CodeRabbit Configration File (`.coderabbit.yaml`)