Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: Metrics do not update when no new queries are routed to the endpoint #173

Open
xqe2011 opened this issue Feb 24, 2025 · 2 comments · May be fixed by #174
Open

bug: Metrics do not update when no new queries are routed to the endpoint #173

xqe2011 opened this issue Feb 24, 2025 · 2 comments · May be fixed by #174
Labels
bug Something isn't working

Comments

@xqe2011
Copy link

xqe2011 commented Feb 24, 2025

Describe the bug

When no new queries are routed to the endpoint, some metrics, qps and num_prefill_requests, are not updated. This bug will break the routingLogic session.

To Reproduce

Start the router, make one request, and watch /metrics, wait any seconds, the qps will always have the same value.

Expected behavior

No response

Additional context

No response
@xqe2011 xqe2011 added the bug Something isn't working label Feb 24, 2025
@gaocegege
Copy link
Collaborator

Thanks for reporting the issue!

@ApostaC
Copy link
Collaborator

ApostaC commented Feb 24, 2025 •
edited
Loading

Thanks for reporting! We've also seen this internally.
cc @Shaoting-Feng

@Shaoting-Feng Shaoting-Feng linked a pull request Feb 24, 2025 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Router] Update qps when no requests Shaoting-Feng/production-stack
3 participants
@gaocegege @xqe2011 @ApostaC