-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: add guide for app cloud testing #25900
Merged
Merged
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
# App <--> Cloud Testing | ||
|
||
Testing communication between the Cypress App and Cypress Cloud can be complex depending on how much data is going back and forth. For example, we recently completed the Debug page feature which involved writing some e2e tests to verify that the page was functioning correctly with the expected data from Cypress Cloud. In this document, we'll share some of our findings from the process of writing these tests for the Debug page. | ||
|
||
## Working with GraphQL requests | ||
|
||
We use GraphQL to interface with Cypress Cloud, so there are some things to know when writing e2e tests between the App and Cloud. | ||
|
||
### GraphQL requests over fetch | ||
|
||
By default, GraphQL queries use web sockets to fetch data. This doesn't let us intercept these messages and return a mock response in the ideal way (using cy.intercept) to get our app into the desired state for the test. We can set the GraphQL client to use fetch rather than WS by adding this hook at the top of our spec file. | ||
|
||
You can see an example of this in [our spec for the Debug page](/packages/app/cypress/e2e/debug.cy.ts#L4). | ||
|
||
```js | ||
Cypress.on('window:before:load', (win) => { | ||
win.__CYPRESS_GQL_NO_SOCKET__ = 'true' | ||
}) | ||
``` | ||
|
||
Note that this does not affect [GraphQL subscriptions](#intercepting-graphql-subscriptions). | ||
|
||
Now that the GraphQL requests to Cypress Cloud are happening over fetch, we can use `cy.intercept` to intercept the requests and do whatever we need to do (in most cases, return a JSON fixture as a response to the request). | ||
|
||
### Creating JSON fixtures from GraphQL requests | ||
|
||
Another benefit of making our GraphQL requests over fetch is that we can easily see the real responses from Cypress Cloud in the Network tab of our browser developer tools. This is especially useful when we have large responses coming back from Cypress Cloud that we want to mock in our tests. | ||
|
||
We can start our fixture by copying the real response from the dev tools response into our fixture and then modifying individual fields to tweak our app state. Then we can intercept the GraphQL request and return our fixture. | ||
|
||
You can see an example of this in [our spec for the Debug page](/packages/app/cypress/e2e/debug.cy.ts#L35). | ||
|
||
```js | ||
cy.intercept('query-Debug', { | ||
fixture: 'debug-Passing/gql-Debug.json', | ||
}) | ||
``` | ||
|
||
### Intercepting GraphQL subscriptions | ||
|
||
Subscriptions always use web sockets, so in order to intercept these we need to do something different. For subscriptions, we need to use `cy.remoteGraphQLIntercept` to check if the operation is related to our subscription and if it is, return our fixture instead. | ||
|
||
You can see an example of this in [our spec for the Debug page](/packages/app/cypress/e2e/debug.cy.ts#L23). | ||
|
||
```js | ||
import RelevantRunsDataSource_RunsByCommitShas from '../fixtures/gql-RelevantRunsDataSource_RunsByCommitShas.json' | ||
|
||
beforeEach(() => { | ||
cy.remoteGraphQLIntercept((obj, _testState, options) => { | ||
if (obj.operationName === 'RelevantRunsDataSource_RunsByCommitShas') { | ||
obj.result.data = options.RelevantRunsDataSource_RunsByCommitShas.data | ||
} | ||
|
||
return obj.result | ||
}, { RelevantRunsDataSource_RunsByCommitShas }) | ||
}) | ||
``` |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have a great understanding of the graphql intercepts but looking through the code I don't think that
cy. remoteGraphQLIntercept
has anything to do with subscriptions but rather the ability to intercept queries made to the cloud that originate from the server.For this example, a subscription wired to
pollForRuns
triggers a cloud query fromRelevantRunsDataSource.ts
. If we want to intercept this query, we need to useremoteGraphQLIntercept
. The subscription is just an implementation detail. If it was hooked up to a mutation rather than a subscription, we would be ok just usingcy.intercept
.I think the distinction is important based on how much coverage we want. In true e2e fashion, if we want to test that our code inside of
data-context
is wired up properly we would leveragecy.remoteGraphQLIntercept
as this would allow the data-context methods to run and the only thing stubbed would be the cloud call.The description of this suggests we could use
cy.remoteGraphQLIntercept
to intercept the specs change subscription but I don't think that is possible (rather, we could reach into data-context viacy.withCtx
and trigger a sub manually).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yeah, I think I misunderstood what was going on when I reviewed Lachlan's PR for these tests. What's happening is that this query is coming from
RelevantRunsDataSource.ts
, socy.intercept
won't work and we need to useremoteGraphQLIntercept
. The fact that it's a subscription doesn't have anything to do with it.I'm confused about this part
So if the mutation originated from the server, we'd still be able to intercept it using
cy.intercept
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry should have been more clear. I meant that if there was a frontend mutation that triggered a cloud query, we could intercept the mutation and stub the result essentially bypassing the cloud query (as the methods defined in data-context for initiating the cloud query would never run). Since this example was wired up to subscriptions, we can't do that.
For the most accurate coverage I'd say we shouldn't rely on
cy.intercept
as that skips the graphql resolvers that are hooked up to our data-context methods, but getting the initial state of data-context setup properly could be messyThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah okay. So is your thought that we should either use
cy.remoteGraphQLIntercept
orcy.withCtx
instead of trying to intercept the requests over fetch withcy.intercept
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would say yes but I don't want to pivot from what we currently have. I think we can make a note mentioning the cons of using
cy.intercept
(we lose coverage of our graphql resolvers and data-context calls) and fix up this section to state the correct definition/usage ofcy.remoteGraphQLIntercept
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think you can
cy.intercept
a subscription since it goes over websockets - this is why I had toremoteIntercept
, though what @ZachJW34 says is true - the main difference isremoteIntercept
is between Server<->Cloud, butintercept
between App andserver
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we can use
cy.intercept
for subscriptions, that sure would be neat.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay I made some updates