integration: direct node connectivity test #1189
Open
+182
−90
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The following helpers are all usable universally, not only for tablets: - SingleTargetLBP, - send_statement_everywhere, - send_unprepared_query_everywhere, so it makes sense to extract them to utils.rs.
The following helpers were renamed to better convey their semantics: - `send_statement_everywhere` -> `execute_prepared_statement_everywhere`, - `send_unprepared_query_everywhere` -> `execute_unprepared_statement_everywhere`.
…y values `execute_unprepared_statement_everywhere` was limited to empty values; now any value list is supported.
`execute_(un)prepared_statement_everywhere` both have similar logic that can be extracted to a generic function. `for_each_target_execute` extraction will be even more important in the next commit, where we make its logic a bit more complex not to `unwrap()` the Sharder.
Before, `for_each_target_execute` would panic on absence of Sharder for a Node. This could be caused by two main reasons: 1) the node was unsharded (e.g., a Cassandra node), 2) there were no open connections to the node, e.g. due to the node being down. In both cases, we prefer not to panic in `for_each_target_execute` but rather assume the node is unsharded and proceed. In the case 1), execution of the request should succeed, whereas in 2) we should get the ConnectionPoolError.
I noticed that we lacked a test that requires direct connectivity to all nodes in the cluster in order to pass. Other tests would often succeed even if only one node (the initial contact point) was directly reachable. An example of a test which has exercised direct connectivity to all nodes is tablets.rs, yet it could be enabled only for non-cloud case, as it uses the proxy.
wprzytula
added
the
area/testing
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
I noticed that we lacked a test that requires direct connectivity to all nodes in the cluster in order to pass. Other tests would often succeed even if only one node (the initial contact point) was directly reachable.
The issue has manifested in testing serverless cloud why working on rustls support: tests would pass even with address translation disabled...
Side note
An example of a test which has exercised direct connectivity to all nodes is
tablets.rs, yet it could be enabled only for non-cloud case, as it uses the proxy.Solution
I wrote a test that iterates though all targets (pairs
(node, shard)) and sends a request directly to them (using a load balancing policy that produces singleton query plans).Bonus
I extracted and refactored some utils from
tablets.rstoutils.rs, so that they can be used by other tests.Pre-review checklist
[ ] I have provided docstrings for the public items that I want to introduce.[ ] I have adjusted the documentation in./docs/source/.[ ] I added appropriateFixes:annotations to PR description.