Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update CI - Remove deprecated APIs from TensorRT backend #7193

Merged
merged 34 commits into from
May 10, 2024
Merged

Update CI - Remove deprecated APIs from TensorRT backend #7193

merged 34 commits into from
May 10, 2024

Conversation

pskiran1
Copy link
Member

@pskiran1 pskiran1 commented May 7, 2024

  • Updated model generation scripts to include dynamic batching support for all TensorRT models.
  • Removed deprecated flags.
  • Eliminated deprecated TensorRT plugin tests and replaced them with new custom plugin tests
  • Discontinued the use of caffe2plan (deprecated) and replaced it with trtexec
  • Added a new script for generating ONNX image models with dynamic batch support, which will be utilized to construct TensorRT engines for test cases
  • Adjusted CI tests accordingly to accommodate the changes in the models and tensorrt_backend

pskiran1 and others added 30 commits April 22, 2024 21:56
@pskiran1
Update CI
8f5db22
@pskiran1
Merge branch 'main' of https://github.com/triton-inference-server/server
c4d7221 
 into spolisetty_debug_extended
@pskiran1
Temporary TENSORRT_IMAGE
073230b
@pskiran1
Plugin
7826252
@pskiran1
Test
f01661d
@pskiran1
Update L0_cuda_graph
876e8be
@pskiran1
[FIXME] Modify later INT32 to INT64
6092685
@pskiran1
Testing
d1f0b4a
@pskiran1
Merge branch 'main' of https://github.com/triton-inference-server/server
bc8101d 
 into spolisetty_trt_v10_debug
@pskiran1
Undo datatype changes int64
db195ac
@pskiran1
Merge branch 'main' of https://github.com/triton-inference-server/server
f75baa5 
 into spolisetty_trt_v10_debug
@pskiran1
Skip caffe2plan
f22fff6
@pskiran1
Undo TRT image name
ec7c2d9
@pskiran1
Update CI
9a69668
@pskiran1
Update CI
e654334
@pskiran1
Merge branch 'main' of https://github.com/triton-inference-server/server
6f0577e 
 into spolisetty_DLIS-6351
@pskiran1
Update CI
77f3d69
@pskiran1
Merge branch 'spolisetty_trt_v10_debug' of https://github.com/triton-…
78b014b 
…inference-server/server into spolisetty_DLIS-6351
@pskiran1
Merge branch 'main' of https://github.com/triton-inference-server/server
a9ea07a 
 into spolisetty_trt_v10_debug
@pskiran1
Update plugin CI
5c3ce02
@pskiran1
Update gen_qa_model_repository
7939a61
@pskiran1
Merge branch 'spolisetty_trt_v10_debug' of https://github.com/triton-…
5936d82 
…inference-server/server into spolisetty_DLIS-6351
@tanmayv25
Removing DIRECT_IO flag from reformat-free I/O
3ced746
@pskiran1
Update
e4c4faf
@pskiran1
Merge branch 'spolisetty_trt_v10_debug' of https://github.com/triton-…
6329c4c 
…inference-server/server into spolisetty_DLIS-6351
@pskiran1
Update copyright
6f9fa55
@pskiran1
Update CI
29f28d3
@pskiran1
Remove caffe2plan.cc
2f65be5
@pskiran1
Undo copyright
14e3eae
@pskiran1
Merge branch 'main' of https://github.com/triton-inference-server/server
74af9b2 
 into spolisetty_DLIS-6351
mc-nv added a commit that referenced this pull request May 7, 2024
@mc-nv
Update CI - Remove deprecated APIs from TensorRT backend #7193
1e6964b
@mc-nv mc-nv changed the base branch from rc-24.05 to r24.05 May 9, 2024 22:42
mc-nv added a commit that referenced this pull request May 10, 2024
@mc-nv
Update CI - Remove deprecated APIs from TensorRT backend #7193
d592446
@mc-nv mc-nv merged commit c922f6b into r24.05 May 10, 2024
3 checks passed
@mc-nv mc-nv deleted the spolisetty_DLIS-6351 branch May 10, 2024 20:56
tanmayv25 added a commit that referenced this pull request May 29, 2024
@pskiran1 @tanmayv25
Update CI - Remove deprecated APIs from TensorRT backend (#7193)
09a33c4 
* Update CI

* Temporary TENSORRT_IMAGE

* Plugin

* Test

* Update L0_cuda_graph

* [FIXME] Modify later INT32 to INT64

* Testing

* Undo datatype changes int64

* Skip caffe2plan

* Undo TRT image name

* Update CI

* Update CI

* Update CI

* Update plugin CI

* Update gen_qa_model_repository

* Removing DIRECT_IO flag from reformat-free I/O

* Update

* Update copyright

* Update CI

* Remove caffe2plan.cc

* Undo copyright

* Fix pre-commit errors

* Support INT64 datatype from shape tensors in test cases

* Update comments

---------

Co-authored-by: tanmayv25 <[email protected]>
tanmayv25 added a commit that referenced this pull request May 29, 2024
@pskiran1 @tanmayv25
Update CI - Remove deprecated APIs from TensorRT backend (#7193)
42bc242 
* Update CI

* Temporary TENSORRT_IMAGE

* Plugin

* Test

* Update L0_cuda_graph

* [FIXME] Modify later INT32 to INT64

* Testing

* Undo datatype changes int64

* Skip caffe2plan

* Undo TRT image name

* Update CI

* Update CI

* Update CI

* Update plugin CI

* Update gen_qa_model_repository

* Removing DIRECT_IO flag from reformat-free I/O

* Update

* Update copyright

* Update CI

* Remove caffe2plan.cc

* Undo copyright

* Fix pre-commit errors

* Support INT64 datatype from shape tensors in test cases

* Update comments

---------

Co-authored-by: tanmayv25 <[email protected]>
tanmayv25 added a commit that referenced this pull request May 29, 2024
@tanmayv25 @mc-nv @pskiran1
Update 'main' post 24.05 release (#7298)
4ce0c22 
* Update README and versions for 2.46.0 / 24.05 (#7196)

* Update README.md 2.46.0 / 24.05 (#7273)

* Update CI - Remove deprecated APIs from TensorRT backend (#7193)

* Update CI

* Temporary TENSORRT_IMAGE

* Plugin

* Test

* Update L0_cuda_graph

* [FIXME] Modify later INT32 to INT64

* Testing

* Undo datatype changes int64

* Skip caffe2plan

* Undo TRT image name

* Update CI

* Update CI

* Update CI

* Update plugin CI

* Update gen_qa_model_repository

* Removing DIRECT_IO flag from reformat-free I/O

* Update

* Update copyright

* Update CI

* Remove caffe2plan.cc

* Undo copyright

* Fix pre-commit errors

* Support INT64 datatype from shape tensors in test cases

* Update comments

---------

Co-authored-by: tanmayv25 <[email protected]>

* Fix the versions

---------

Co-authored-by: Misha Chornyi <[email protected]>
Co-authored-by: Sai Kiran Polisetty <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanmayv25 tanmayv25 tanmayv25 approved these changes

@Tabrizian Tabrizian Awaiting requested review from Tabrizian

@GuanLuo GuanLuo Awaiting requested review from GuanLuo

@mc-nv mc-nv Awaiting requested review from mc-nv

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pskiran1 @tanmayv25 @mc-nv