Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util/log: prevent ReportPanic() from swallowing panics in some cases #52201

Merged
merged 1 commit into from
Aug 3, 2020

Conversation

knz
Copy link
Contributor

@knz knz commented Jul 31, 2020

Accompanies #52200 (but is an independent change)

Prior to this patch, if ReportPanic() was called and the panic
object ultimately discarded (ie. caught) during tests that were not otherwise using
TestLogScope, the panic object would be lost. I believe this was directly noticed by @andreimatei prior.

I also believe this was causing panic objects to disappear quite a lot in fact,
because the test runner also catches panics and fails to report
them adequately in some cases (e.g. when the panic occurs during a
stress run).

To alleviate the situation, this patch removes some logic previously
present in the code that was attempting (and failing) to remove
duplicate panic prints. That logic was misdesigned to start with anyway
(by me), because it was working under the assumption that
ReportPanic() was only called for uncaught panics, and another
function RecoverAndReportNonfatalPanic() just before that was
blatantly violating that assumption.

Removing the logic causes reportable panics to always be reported in
logs and on stderr, regardless of whether it's caught or not. This
provides more guarantees that the panic object will be seen, at the
expense of having a duplicate print in some edge cases.

Release note: None

@knz knz requested review from andreimatei and irfansharif July 31, 2020 19:49
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Prior to this patch, if `ReportPanic()` was called and the panic
object ultimately discarded (ie. caught) during tests not otherwise using
`TestLogScope`, the panic object would be lost.

This was causing panic objects to disappear quite a lot in fact,
because the test runner *also* catches panics and fails to report
them adequately in some cases (e.g. when the panic occurs during a
`stress` run).

To alleviate the situation, this patch removes some logic previously
present in the code that was attempting (and failing) to remove
duplicate panic prints. That logic was misdesigned to start with anyway
(by me), because it was working under the assumption that
`ReportPanic()` was *only* called for uncaught panics, and another
function `RecoverAndReportNonfatalPanic()` just before that was
blatantly violating that assumption.

Removing the logic causes reportable panics to always be reported in
logs and on stderr, regardless of whether it's caught or not. This
provides more guarantees that the panic object will be seen, at the
expense of having a duplicate print in some edge cases.

Release note: None
@knz knz force-pushed the 20200731-double-panic branch from 72e090b to e9a159b Compare August 1, 2020 08:49
@knz knz requested a review from a team as a code owner August 1, 2020 08:49
Copy link
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andreimatei and @irfansharif)

@knz
Copy link
Contributor Author

knz commented Aug 3, 2020

thanks!

bors r=andreimatei

@craig
Copy link
Contributor

craig bot commented Aug 3, 2020

Build succeeded:

@craig craig bot merged commit e36a617 into cockroachdb:master Aug 3, 2020
@knz knz deleted the 20200731-double-panic branch August 3, 2020 18:47
knz added a commit to knz/cockroach that referenced this pull request Aug 25, 2020
This flake was introduced in a recent bump of `cockroachdb/errors`,
combined with cockroachdb#52201.

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants