chore: add guide for app cloud testing #25900
Conversation
27 flaky tests on run #44301 ↗︎
Details:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Test | ||
|---|---|---|
| network stubbing > intercepting request > can delay and throttle a StaticResponse | ||
cypress/cypress.cy.js • 3 flaky tests • 5x-driver-firefox
| Test | ||
|---|---|---|
| ... > correctly returns currentRetry | ||
| ... > correctly returns currentRetry | ||
| ... > correctly returns currentRetry | ||
create-from-component.cy.ts • 2 flaky tests • app-e2e
| Test | ||
|---|---|---|
| ... > runs generated spec |
Screenshot
|
|
| ... > runs generated spec |
Screenshot
|
|
specs_list_latest_runs.cy.ts • 1 flaky test • app-e2e
| Test | ||
|---|---|---|
| App/Cloud Integration - Latest runs and Average duration > when no runs are recorded > shows placeholders for all visible specs |
Screenshot
|
|
cypress-origin-communicator.cy.ts • 1 flaky test • app-e2e
| Test | ||
|---|---|---|
| Cypress In Cypress Origin Communicator > cy.origin passivity with app interactions > passes upon test reload mid test execution |
Screenshot
|
|
The first 5 flaky specs are shown, see all 14 specs in Cypress Cloud.
This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.
guides/app-cloud-testing.md
Outdated
| }) | ||
| ``` | ||
|
|
||
| ### Intercepting GraphQL subscriptions |
There was a problem hiding this comment.
I don't have a great understanding of the graphql intercepts but looking through the code I don't think that cy. remoteGraphQLIntercept has anything to do with subscriptions but rather the ability to intercept queries made to the cloud that originate from the server.
For this example, a subscription wired to pollForRuns triggers a cloud query from RelevantRunsDataSource.ts. If we want to intercept this query, we need to use remoteGraphQLIntercept. The subscription is just an implementation detail. If it was hooked up to a mutation rather than a subscription, we would be ok just using cy.intercept.
I think the distinction is important based on how much coverage we want. In true e2e fashion, if we want to test that our code inside of data-context is wired up properly we would leverage cy.remoteGraphQLIntercept as this would allow the data-context methods to run and the only thing stubbed would be the cloud call.
The description of this suggests we could use cy.remoteGraphQLIntercept to intercept the specs change subscription but I don't think that is possible (rather, we could reach into data-context via cy.withCtx and trigger a sub manually).
There was a problem hiding this comment.
Ah yeah, I think I misunderstood what was going on when I reviewed Lachlan's PR for these tests. What's happening is that this query is coming from RelevantRunsDataSource.ts, so cy.intercept won't work and we need to use remoteGraphQLIntercept. The fact that it's a subscription doesn't have anything to do with it.
I'm confused about this part
If it was hooked up to a mutation rather than a subscription, we would be ok just using
cy.intercept.
So if the mutation originated from the server, we'd still be able to intercept it using cy.intercept?
There was a problem hiding this comment.
Sorry should have been more clear. I meant that if there was a frontend mutation that triggered a cloud query, we could intercept the mutation and stub the result essentially bypassing the cloud query (as the methods defined in data-context for initiating the cloud query would never run). Since this example was wired up to subscriptions, we can't do that.
For the most accurate coverage I'd say we shouldn't rely on cy.intercept as that skips the graphql resolvers that are hooked up to our data-context methods, but getting the initial state of data-context setup properly could be messy
There was a problem hiding this comment.
Ah okay. So is your thought that we should either use cy.remoteGraphQLIntercept or cy.withCtx instead of trying to intercept the requests over fetch with cy.intercept?
There was a problem hiding this comment.
I would say yes but I don't want to pivot from what we currently have. I think we can make a note mentioning the cons of using cy.intercept (we lose coverage of our graphql resolvers and data-context calls) and fix up this section to state the correct definition/usage of cy.remoteGraphQLIntercept
There was a problem hiding this comment.
I don't think you can cy.intercept a subscription since it goes over websockets - this is why I had to remoteIntercept, though what @ZachJW34 says is true - the main difference is remoteIntercept is between Server<->Cloud, but intercept between App and server.
There was a problem hiding this comment.
If we can use cy.intercept for subscriptions, that sure would be neat.
There was a problem hiding this comment.
Okay I made some updates
Additional details
This PR adds a guide that describes how to work with GraphQL stuff while writing e2e tests between App and Cloud
PR Tasks
cypress-documentation?type definitions?