[serve] recursive cancellation #47935
Labels
P1
Issue that should be fixed within a few weeks
ray-team-created
Ray Team created
serve
Ray Serve Related Issue
We've relied on recursive cancellation of tasks in Ray Core, but that doesn't fully cover recursive cancellation of requests in Ray Serve. If a request is queued and hasn't been sent off to a replica yet at the time of the parent request's cancellation, then it will remain in the queue and eventually be sent to a downstream replica, i.e. the recursive cancellation will fail.
We should fix this by implementing recursive cancellation at the Serve layer.
The text was updated successfully, but these errors were encountered: