Reinstate and skip Dashboard and Visualize tests #15687
Conversation
cjcenizal
force-pushed
the
skip-dashboard-tests
branch
from
99b5017 to
9c8d3d3
Compare
* Reinstate and skip flaky Dashboard view edit tests. * Reinstate and skip flaky Visualize gauge chart tests.
| * Tests add time to the development cycle so we should write as few tests as possible, but no fewer. | ||
| * **Unit tests** should cover 70% of our needs. They'll ensure our fundamentals are sound. | ||
| * **Integrations tests** should cover 20% of our needs. They'll ensure our units work well together. | ||
| * **End-to-end tests** should cover 10% of our needs. They'll ensure our primary user flows are intact and alleviate the burden of manual testing from our QA engineers. |
There was a problem hiding this comment.
I feel a bit like these numbers are arbitrary and don't accurately reflect our needs given the state of our code base. End to end tests require updating when our UI/UX changes, but they don't require updating when the inner code changes. Unit tests and integration tests require updating when our code changes, but not when the UI/UX changes. If we had a more stable code base then I think I'd agree that unit tests should be the focus, but we don't. Many of the bugs we actually find in production can't be covered by a unit tests. Unit tests aren't easy to create if the code doesn't have clear boundaries between logic and UI, which many places of our code base don't. In addition, a large portion of our codebase is in a transition from angular to react and undergoing a lot of clean up and refactoring. We should be writing unit tests for new code where possible, but I think code thats undergoing a big refactor means it's even more important to have end to end tests to ensure all the corner cases still work. It also eases the burden on our QA team.
I think a good principle to have is that any time a bug is encountered, write a test that would have caught it, whether it be a unit tests or an integration test, or an end to end test, but don't change the code prior to adding the test. I've been trying to do this and most of the time the only way to cover the test is an end to end test.
This reinstates some tests removed by #15657