Skip to content

Commit

Permalink
Add note on --cache-config spacing and fix typos (#6929)
Browse files Browse the repository at this point in the history
  • Loading branch information
rmccorm4 authored Mar 1, 2024
1 parent 5630efe commit ac5ba42
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 5 deletions.
2 changes: 1 addition & 1 deletion docs/user_guide/model_configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -723,7 +723,7 @@ The above configuration creates 3 model instances, one on each device
themselves as "R1" is local for their own device, however, they will
contend for "R2" because it is specified as a global resource which
means "R2" is shared across the system. Though these instances don't
contend for "R1" among themsleves, but they will contend for "R1"
contend for "R1" among themselves, but they will contend for "R1"
with other model instances which includes "R1" in their resource
requirements and run on the same device as them.

Expand Down
10 changes: 7 additions & 3 deletions docs/user_guide/response_cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,16 +58,20 @@ sections below for more details.

### Enable Caching on Server-side

The response cache is enabled on the server-side by specifying a
`<cache_implementation>` and corresponding configuration when starting
The response cache is enabled on the server-side by specifying a cache
implementation name `<cache>` and corresponding configuration when starting
the Triton server.

Through the CLI, this translates to setting
`tritonserver --cache-config <cache_implementation>,<key>=<value> ...`. For example:
`tritonserver --cache-config <cache>,<key>=<value> ...`. For example:
```
tritonserver --cache-config local,size=1048576
```

> [!NOTE]
> If using a non-interactive shell, you may need to specify the argument without
> the space like so: `--cache-config=<cache>,<key>=<value>`.
For in-process C API applications, this translates to calling
`TRITONSERVER_SetCacheConfig(const char* cache_implementation, const char* config_json)`.

Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/trace.md
Original file line number Diff line number Diff line change
Expand Up @@ -543,7 +543,7 @@ The following table shows available OpenTelemetry trace APIs settings for
For example:<br/>
<code>--trace-config opentelemetry,resource=service.name=triton</code><br/>
<code>--trace-config opentelemetry,resource=service.version=1</code><br/>
Alternatively, key-vaue attributes can be specified through <br/>
Alternatively, key-value attributes can be specified through <br/>
<a href="https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes">
OTEL_RESOURCE_ATTRIBUTES</a>
environment variable.
Expand Down

0 comments on commit ac5ba42

Please sign in to comment.