Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[serve] recursive cancellation #47935

Closed
zcin opened this issue Oct 8, 2024 · 1 comment
Closed

[serve] recursive cancellation #47935

zcin opened this issue Oct 8, 2024 · 1 comment
Assignees
Labels
P1 Issue that should be fixed within a few weeks ray-team-created Ray Team created serve Ray Serve Related Issue

Comments

@zcin
Copy link
Contributor

zcin commented Oct 8, 2024

We've relied on recursive cancellation of tasks in Ray Core, but that doesn't fully cover recursive cancellation of requests in Ray Serve. If a request is queued and hasn't been sent off to a replica yet at the time of the parent request's cancellation, then it will remain in the queue and eventually be sent to a downstream replica, i.e. the recursive cancellation will fail.

We should fix this by implementing recursive cancellation at the Serve layer.

@zcin zcin self-assigned this Oct 8, 2024
@zcin zcin added serve Ray Serve Related Issue ray-team-created Ray Team created labels Oct 8, 2024
@akshay-anyscale akshay-anyscale added the P1 Issue that should be fixed within a few weeks label Oct 22, 2024
@zcin
Copy link
Contributor Author

zcin commented Oct 29, 2024

closed by #47873

@zcin zcin closed this as completed Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 Issue that should be fixed within a few weeks ray-team-created Ray Team created serve Ray Serve Related Issue
Projects
None yet
Development

No branches or pull requests

2 participants