Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Remove chan close in rehydration receiver shutdowns #2128

Conversation

dpaasman00
Copy link
Contributor

Proposed Change

Closing the done channel in shutdown was causing the function to block and not shutdown properly. I believe this is because we are calling the ctx cancel func which is going to cause the go routine to return before it can read from the done chan.

Also acquire the lock before closing the checkpoint store.

Checklist
  • Changes are tested
  • CI has passed

@dpaasman00 dpaasman00 requested a review from a team as a code owner January 29, 2025 14:06
@dpaasman00 dpaasman00 requested review from justinianvoss22 and schmikei and removed request for justinianvoss22 January 29, 2025 14:06
@schmikei
Copy link
Contributor

For some reason I'm getting this error even with your change:

{"level":"error","ts":"2025-01-29T09:40:02.778-0500","caller":"azureblobrehydrationreceiver/receiver.go:305","msg":"Error while saving checkpoint","kind":"receiver","name":"azureblobrehydration","data_type":"logs","error":"database not open","stacktrace":"github.com/observiq/bindplane-otel-collector/receiver/azureblobrehydrationreceiver.(*rehydrationReceiver).rehydrateBlobs\n\t/Users/keithschmitt/workspace/bindplane-agent/receiver/azureblobrehydrationreceiver/receiver.go:305\ngithub.jparrowsec.cn/observiq/bindplane-otel-collector/receiver/azureblobrehydrationreceiver.(*rehydrationReceiver).streamRehydrateBlobs\n\t/Users/keithschmitt/workspace/bindplane-agent/receiver/azureblobrehydrationreceiver/receiver.go:239"}

🤔

@dpaasman00 dpaasman00 force-pushed the dakotapaasman/bpop-1804-s3-rehydration-bad-shutdown-behavior branch from 270cd88 to 4cb3024 Compare January 30, 2025 16:10
@dpaasman00 dpaasman00 merged commit 4560141 into release/v1.70.0 Jan 30, 2025
15 checks passed
@dpaasman00 dpaasman00 deleted the dakotapaasman/bpop-1804-s3-rehydration-bad-shutdown-behavior branch January 30, 2025 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants