Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix version for setuptools and grpcio-tools. Remove cudnn 8 installation #7331

Merged
merged 6 commits into from
Jun 18, 2024

Conversation

krishung5
Copy link
Contributor

@krishung5 krishung5 commented Jun 6, 2024

What does the PR do?

Fixed the version for setuptools to avoid breaking changes in the later versions.

Checklist

  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • build
  • ci
  • docs
  • feat
  • fix
  • perf
  • refactor
  • revert
  • style
  • test

Related PRs:

Where should the reviewer start?

Test plan:

L0_backend_trtllm* tests should be passing.

  • CI Pipeline ID: 15903582

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@krishung5 krishung5 added PR: ci Changes to our CI configuration files and scripts PR: fix A bug fix labels Jun 6, 2024
@krishung5 krishung5 requested a review from mc-nv June 6, 2024 18:56
mc-nv
mc-nv previously approved these changes Jun 6, 2024
@krishung5 krishung5 force-pushed the krish-fix-trtllm-test branch from f435e89 to 6f85431 Compare June 11, 2024 19:13
@krishung5 krishung5 changed the title fix: Fix version for setuptools and grpcio-tools fix: Fix version for setuptools Jun 11, 2024
@krishung5 krishung5 requested a review from tanmayv25 June 11, 2024 23:31
tanmayv25
tanmayv25 previously approved these changes Jun 12, 2024
@krishung5 krishung5 changed the title fix: Fix version for setuptools fix: Fix version for setuptools and grpcio-tools. Remove cudnn 8 installation Jun 18, 2024
@krishung5 krishung5 requested a review from tanmayv25 June 18, 2024 00:44
@krishung5
Copy link
Contributor Author

krishung5 commented Jun 18, 2024

Removed cudnn 8 installation since the newer Pytorch base image doesn't have issues with cudnn 8. Updated pipeline ID.
CC @nv-kmcgill53

nv-kmcgill53
nv-kmcgill53 previously approved these changes Jun 18, 2024
RUN find /usr -name libtensorrt_llm.so -exec dirname {} \; > /etc/ld.so.conf.d/tensorrt-llm.conf
RUN find /opt/tritonserver -name libtritonserver.so -exec dirname {} \; > /etc/ld.so.conf.d/triton-tensorrtllm-worker.conf

RUN pip3 install setuptools==69.5.1 grpcio-tools==1.64.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any particular reason for pinning the versions here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setuptools has some breaking changes of pkg_resources in the latest 70.0.0 version. See here for more info.
The fixed version for grpcio-tools is to resolve the below error:

python3 /opt/tritonserver/tensorrtllm_backend/inflight_batcher_llm/client/end_to_end_grpc_client.py '--prompt=My name is'
/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/service_pb2_grpc.py:21: RuntimeWarning: The grpc package installed is at version 1.62.1, but the generated code in grpc_service_pb2_grpc.py depends on grpcio>=1.64.0. Please upgrade your grpc module to grpcio>=1.64.0 or downgrade your generated code using grpcio-tools<=1.62.1. This warning will become an error in 1.65.0, scheduled for release on June 25, 2024.
  warnings.warn(
client creation failed: Channel.unary_unary() got an unexpected keyword argument '_registered_method'
Exception ignored in: <function InferenceServerClient.__del__ at 0xfffeef103c70>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/_client.py", line 257, in __del__
    self.close()
  File "/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/_client.py", line 264, in close
    self.stop_stream()
  File "/usr/local/lib/python3.10/dist-packages/tritonclient/grpc/_client.py", line 1811, in stop_stream
    if self._stream is not None:
AttributeError: 'InferenceServerClient' object has no attribute '_stream'

I have added the reason to the script.

@krishung5 krishung5 merged commit f43cedd into main Jun 18, 2024
3 checks passed
@krishung5 krishung5 deleted the krish-fix-trtllm-test branch June 18, 2024 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR: ci Changes to our CI configuration files and scripts PR: fix A bug fix
Development

Successfully merging this pull request may close these issues.

4 participants