-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add trace mode and trace config entries in trace settings API #7050
Conversation
Please update PR title to be more descriptive for the changes 🙏 |
|
auto mode_key = std::to_string(trace_mode); | ||
auto trace_options_it = config_map.find(mode_key); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Food for thought in the future with Olga, but I found this code misleading to read.
Upon reading it, I naively thought std::to_string(trace_mode)
would be something like "TRITON"
or "OPENTELEMETRY"
, but it's actually "0" or "1" based on order in the enum. InferenceTraceModeString
is the one that actually converts to a string.
TLDR; I found it confusing that the keys in this map are "0" and "1" rather than the actual string versions of the modes. The keys in the map could probably just be the enums themselves (TRACE_MODE_TRITON or TRACE_MODE_OPENTELEMETRY [interpreted as integer 0 or 1] rather than string 0 or 1), or the actual string representations ("triton" and "opentelemtry").
No action here, just mentioning something I found misleading to think about down the line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice changes! 🚀
request.model_name(), &level, &rate, &count, &log_frequency, &filepath, | ||
&trace_mode, &config_map); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is probably due for some cleanup in the near future, where you can encapsulate this growing list of parameters into a struct/class or something if makes sense like:
GetTraceSetting(model_name, &trace_settings)
rather than all the individual args.
However, I think this can be a follow-up cleanup rather than a blocker for this change. Feel free to add a JIRA ticket if you agree.
I think some other sections could probably be condensed or cleaned up as well if encapsulating the fields in an object.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes I will take this up in the Update PR for
https://jirasw.nvidia.com/browse/DLIS-4795
Random note @indrajit96 but I also see the message in this block (not from your changes) is wrong. The warning should mention deprecrated Feel free to add to this PR, or submit a separate small message-fix PR |
@@ -64,6 +64,7 @@ namespace triton { namespace server { | |||
|
|||
using TraceConfig = std::vector< | |||
std::pair<std::string, std::variant<std::string, int, uint32_t>>>; | |||
// Key is trace mode, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not only trace mode, IIRC, for global settings the key is an empty string
As a follow-up, I think it is nice to send to the user only relevant settings, i.e. if mode is Next suggestion, can we lowercase mode in the response, please? When users set up mode, it is in lower case, so it would be nice to send it back in lower case as well |
Resolved |
Resolved |
Co-authored-by: Ryan McCormick <[email protected]>
…/server into indrajit_tracing
…/server into indrajit_tracing
Add trace_mode and trace_config to getTraceSettingsAPI --------- Co-authored-by: Ryan McCormick <[email protected]>
…#7103) Add trace_mode and trace_config to getTraceSettingsAPI --------- Co-authored-by: Ryan McCormick <[email protected]>
PR for
https://jirasw.nvidia.com/browse/DLIS-4960
/trace/setting API updated to return
Old output:
{"trace_level":["TIMESTAMPS"],"trace_rate":"1","trace_count":"-1","log_frequency":"0","trace_file":""}
New Output:
{"trace_level":["TIMESTAMPS"],"trace_rate":"1","trace_count":"-1","log_frequency":"0","trace_file":"","trace_mode":"OPENTELEMETRY","url":"localhost:4318/v1/traces","bsp_max_queue_size":"1","bsp_max_export_batch_size":"512","bsp_schedule_delay":"5000"}
Tested for both modes TRITON and OPENTELEMETRY.