-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ensure workers can send data popped off the queue at shutdown #239
Ensure workers can send data popped off the queue at shutdown #239
Conversation
Previously, any data still in the queue would be handled by the main thread. But, any data that a worker had already popped off the queue, but not yet sent on the wire, would be lost. This change pushes down the logic for stopping into the writer and worker classes. The worker tells the threads to exit and waits for them to do so.
My last attempt to address this issue in #199 only got part way there. That solved the problem of data being left on the |
The builds that failed, failed with:
It seems to me like with an async client, it's possible for two different workers to pick up the two datapoints written to the queue, and they wouldn't be able to aggregate them. Should this really be a test? Or maybe we could keep the test, but set the number of workers to 1? The tests succeed on my laptop. |
@dmke Have you had a chance to look at this? |
@onlynone sorry for the delay on this one. Will take a look at this when I get a chance |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Merged in 8df479a. Thank you! |
Previously, any data still in the queue would be handled by the main
thread. But, any data that a worker had already popped off the queue,
but not yet sent on the wire, would be lost.
This change pushes down the logic for stopping to the writer and
worker which now tell the threads to exit and wait for them to do so.