-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Fix version for setuptools and grpcio-tools. Remove cudnn 8 installation #7331
Conversation
f435e89
to
6f85431
Compare
Removed cudnn 8 installation since the newer Pytorch base image doesn't have issues with cudnn 8. Updated pipeline ID. |
RUN find /usr -name libtensorrt_llm.so -exec dirname {} \; > /etc/ld.so.conf.d/tensorrt-llm.conf | ||
RUN find /opt/tritonserver -name libtritonserver.so -exec dirname {} \; > /etc/ld.so.conf.d/triton-tensorrtllm-worker.conf | ||
|
||
RUN pip3 install setuptools==69.5.1 grpcio-tools==1.64.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any particular reason for pinning the versions here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Setuptools has some breaking changes of pkg_resources
in the latest 70.0.0 version. See here for more info.
The fixed version for grpcio-tools is to resolve the below error:
python3 /opt/tritonserver/tensorrtllm_backend/inflight_batcher_llm/client/end_to_end_grpc_client.py '--prompt=My name is'
/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/service_pb2_grpc.py:21: RuntimeWarning: The grpc package installed is at version 1.62.1, but the generated code in grpc_service_pb2_grpc.py depends on grpcio>=1.64.0. Please upgrade your grpc module to grpcio>=1.64.0 or downgrade your generated code using grpcio-tools<=1.62.1. This warning will become an error in 1.65.0, scheduled for release on June 25, 2024.
warnings.warn(
client creation failed: Channel.unary_unary() got an unexpected keyword argument '_registered_method'
Exception ignored in: <function InferenceServerClient.__del__ at 0xfffeef103c70>
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/_client.py", line 257, in __del__
self.close()
File "/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/_client.py", line 264, in close
self.stop_stream()
File "/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/_client.py", line 1811, in stop_stream
if self._stream is not None:
AttributeError: 'InferenceServerClient' object has no attribute '_stream'
I have added the reason to the script.
What does the PR do?
Fixed the version for setuptools to avoid breaking changes in the later versions.
Checklist
<commit_type>: <Title>
Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
Where should the reviewer start?
Test plan:
L0_backend_trtllm* tests should be passing.
Caveats:
Background
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)