Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[multistage] Follow-ups for Multistage Mailbox Stability #10424

Closed
7 tasks
ankitsultana opened this issue Mar 15, 2023 · 0 comments
Closed
7 tasks

[multistage] Follow-ups for Multistage Mailbox Stability #10424

ankitsultana opened this issue Mar 15, 2023 · 0 comments
Labels
multi-stage Related to the multi-stage query engine

Comments

@ankitsultana
Copy link
Contributor

ankitsultana commented Mar 15, 2023

I recently raised a few PRs to fix a bunch of stability issues in the multistage engine.

There are still some issues pending though. Using this issue to track them.

  • Complete the design doc for the changes in https://github.com/apache/pinot/pull/10322/files
  • Address out of order cancellation for both in-memory and gRPC mailbox.
  • Handle gRPC sending mailbox cancellation when stream is created but no message has been sent (I think there's a case where only onCompleted is called and onNext is never called).
  • Handle gRPC sending mailbox cancellation when stream is never created so the receiver never gets any notification. A fix for this was added in [multistage] Handle Stream Cancellations for Unstarted Streams #10425 but we should discuss this in the design doc and see whether that solution needs to be improved.
  • Ensure error message is sent upstream in case of cancellations. This PR added a potential fix for this but there are some more cases: [multistage] Handle Stream Cancellations for Unstarted Streams #10425
  • Add UTs which use MultiplexingMailboxService. Right now QueryRunnerTest uses only gRPC.
  • Add Callbacks for complete events.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
multi-stage Related to the multi-stage query engine
Projects
None yet
Development

No branches or pull requests

2 participants