Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: cdc/sink-chaos failed #91384

Closed
cockroach-teamcity opened this issue Nov 7, 2022 · 37 comments
Closed

roachtest: cdc/sink-chaos failed #91384

cockroach-teamcity opened this issue Nov 7, 2022 · 37 comments
Assignees
@samiskin
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. T-cdc
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Nov 7, 2022 •
edited by exalate-issue-sync bot
Loading

roachtest.cdc/sink-chaos failed with artifacts on master @ 4c828ca5df8ea70b92ba58c6cb4868303cee1262:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m19.020031785s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

/cc @cockroachdb/cdc

This test on roachdash | Improve this report!

Jira issue: CRDB-21240

Epic CRDB-11732
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Nov 7, 2022
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Nov 7, 2022
@blathers-crl blathers-crl bot added the T-cdc label Nov 7, 2022
@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ ff8f7eda167887e1f11bbcd9029aa4cc2d28085e:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 15m29.647697853s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ ee4fdb7fff683a7916ee585ac0c1e22d5d9a4412:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 16m28.028763503s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ d6ae9653b67071cca09ed296711cecc821fc284e:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m19.764671262s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ d136c1fa50fa507c2232d7b215d98b88a83649c5:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m53.339221835s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ fb4014a31b9b8d8235dc48de52196e64b185f490:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m18.361354237s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ af18b5d2f8ec09d6f3c3092be9ef5fe3f460724d:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m21.490518667s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 7f77f57a768a1bc488629234126e9e456bb6ddbb:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m23.595239403s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@miretskiy
Copy link
Contributor

The timing of when this failure started happening is suspiciously close to #90810 merging.
One of the things above PR did was to make retries less aggressive by increasing sleep
between subsequent failures to be longer. Perhaps this part needs to be tweaked for chaos testing.
Regardless, given that this failure does not occur on 22.2 rel branch, I'm going to drop release-blocker label.

@miretskiy miretskiy removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Nov 14, 2022
@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 47c7b3a1bc047fc3e481cf12166885b39519c022:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m24.20608514s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ d98d195dfa7a3480ce3e07657aa870692fe71cea:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m45.587857972s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ ea7c52e5a930be203318db6e338edd5dde7537bd:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m26.203885884s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ a035d2e6e7ea57b30115ecc8ee1a5d553a9e3412:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 8m24.832155501s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 8357abb668a5adaff781343b394b162fb1b66c6e:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 17m37.006641125s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ cfb5ae9a96e1770daa4aef1615a46e212b561a84:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 7m2.654470861s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ f694132d6ed73c707576a28f6e784fda8ea13c63:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 8m25.913607973s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ dca415eddac0d659ae6d76b4e3dfdf4076adbd34:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m28.901183573s vs 5m0s

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 1a6e9f885baa124d5ff2996adb966ea15a1a9b2b:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 15m26.386817282s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 1a6e9f885baa124d5ff2996adb966ea15a1a9b2b:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 7m52.009292417s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 1a6e9f885baa124d5ff2996adb966ea15a1a9b2b:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 11m48.217541352s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 2b595c5214aead3e7d4f47f9a1dedce57dba6e7f:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 17m38.47298171s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 7823e2f75474417a44f0d34f76c5f8914cb91d52:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 11m21.287611174s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ e7b15ebaed9c14668ade0a7827a5525aedef1ab0:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 11m43.637667683s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 9c21578450e395c83a1dc0df7090296fef06e006:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 15m34.127681054s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 8a5cb5109d6b7e122e996a44f940bfd61698cd81:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 11m40.818343318s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 146556e19f5e4fdc8c3e6a623b280cc33aee4d18:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m31.467953103s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 8165e3974c10e88b6ae11c6255872ea16f3a67e3:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 17m34.108955787s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity cockroach-teamcity mentioned this issue Dec 8, 2022
Closed
@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ ec095bc2fdbe4e518b076db20e4920fab67222bf:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m24.98016871s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 24854994805cede37e6845ee2a94e10272b5506b:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 11m0.180001132s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ f2b00e8039af6ea8887ec124dad8daf19da6fbf1:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m16.452006089s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ c050c9b4b57ecf2ceb5d449c31c617fe12c920e0:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 10m54.243496416s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 942a4d468e9c8ad0ef45a7be33f0a326dfb19fef:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m38.945656489s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ a80652b2e4691ea76ea49e797b1b9e0998e1d61f:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m10.692873898s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ a80652b2e4691ea76ea49e797b1b9e0998e1d61f:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 13m39.633786022s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ 052acc88ad9d7296ce6b8b441627fb469cc74d95:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 9m21.188975986s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.cdc/sink-chaos failed with artifacts on master @ d6f98e90684894fd36f53596e6aac355676d232e:

test artifacts and logs in: /artifacts/cdc/sink-chaos/run_1
(test_impl.go:297).Fatalf: max latency was more than allowed: 11m27.169639129s vs 5m0s
(test_impl.go:291).Fatal: monitor failure: monitor task failed: t.Fatal() was called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity cockroach-teamcity mentioned this issue Dec 16, 2022
Closed
@miretskiy
Copy link
Contributor

Your changes to chaos test are in? Can this be closed?

@samiskin
Copy link
Contributor

Yep they're in now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@samiskin samiskin

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. T-cdc
Projects
None yet
Milestone
23.1
Development

No branches or pull requests
3 participants
@samiskin @cockroach-teamcity @miretskiy