-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
backupccl: replace restore2TB and restoretpccInc tests #98072
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
overall makes sense, a few things:
- you're also dropping a tpcc roachperf, right? probably intentional, mention it in the pr/commit description?
- in roachperf we have 2 bank tests with 6 nodes, one has no data, I'm guessing it was removed from restore.go but not from roachperf? or something like that.. which brings me to the question - do you need to remove those from roachperf also?
- the 32 nodes bank test [1] shows a regression since last Dec, we're going to lose this info now. The newly added tests here don't backfill, so maybe we should keep those tests for a few more months to have an overlap with the new tests? WDYT?
[1] https://roachperf.crdb.dev/?filter=restore&view=restore2TB%2Fnodes%3D32&tab=gce
to paraphrase your last question: " should we keep them around for a couple months for perf regression detection?"
|
I think it is helpful to have it, it helps us understand that things regressed slowly in that time range. I know it's not the best test but it's info I'd like to have available for debugging. anyway, sounds like we'll have the history so we're good, thank you! |
This patch removes the restore2TB* roachtests which ran a 2TB bank restore to benchmark restore performance across a few hardware configurations. This patch also replaces the `restoreTPCCInc/nodes=10` test which tested our ability to handle a backup with a long chain. This patch also adds: 1. `restore/tpce/400GB/aws/nodes=4/cpus=16` to measure how per-node throughput scales when the per node vcpu count doubles relative to default. 2. `restore/tpce/400GB/aws/nodes=8/cpus=8` to measure how per-node throughput scales when the number of nodes doubles relative to default. 3. `restore/tpce/400GB/aws/backupsIncluded=48/nodes=4/cpus=8` to measure restore reliability and performance on 48 length long backup chain relative to default. A future patch will update the fixtures used in the restore node shutdown scripts, and add more perf based tests. Fixes cockroachdb#92699 Release note: None
0296cb2
to
b588f1f
Compare
TFTR! bors r=lidorcarmel |
Build succeeded: |
This patch removes the restore2TB* roachtests which ran a 2TB bank restore to
benchmark restore performance across a few hardware configurations. This patch
also replaces the
restoreTPCCInc/nodes=10
test which tested our ability tohandle a backup with a long chain.
This patch also adds:
restore/tpce/400GB/aws/nodes=4/cpus=16
to measure how per-node throughputscales when the per node vcpu count doubles relative to default.
restore/tpce/400GB/aws/nodes=8/cpus=8
to measure how per-node throughputscales when the number of nodes doubles relative to default.
restore/tpce/400GB/aws/backupsIncluded=48/nodes=4/cpus=8
to measurerestore reliability and performance on 48 length long backup chain relative to
default.
A future patch will update the fixtures used in the restore node shutdown
scripts, and add more perf based tests.
Fixes #92699
Release note: None