feat: C++ runtime on Windows

[HolyWu]

Description

Fix and enable C++ runtime on Windows.

Fixes #2247
Fixes #2371
Fixes #2484

Type of change

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[HolyWu]

Some observations about the failed tests.

• Test torchscript frontend: TypeError: convert_graph_to_trt_engine(): incompatible function arguments and TypeError: compile_graph(): incompatible function arguments. Caused by ABI incompatibility with pybind11. Fixed in a later commit.
• Test dynamo converters: ValueError: [TRT] [E] Could not implicitly convert NumPy data type: i32 to TensorRT. Introduced by de81be2. Replacing dtype=np.dtype("i") with dtype=np.int32 can fix it. But I'm not sure whether np.dtype("i") always equals int32 or int64 depending on platforms. So I'll leave this to you.
• Test dynamo export serde: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/trt.ep'. Should just use "./trt.ep" instead of "/tmp/trt.ep" like the other tests. Fixed in a later commit.

[gs-olive]

This looks great and thank you very much for the contribution. Added a few questions/comments. The comments on the test cases + added fixes look good and switching np.dtype("i") to np.int32 for that case is fine to include in the converter.

[pytorch-bot]

No ciflow labels are configured for this repo.
For information on how to enable CIFlow bot see this wiki

[gs-olive]

Hi @HolyWu - I wanted to share a few notes on the C++ runtime on Windows after some local testing:

• When compiling locally, I had to upgrade the Bazel version to 6.3.2 so it could find cl and other C++ WIndows dependencies. Still, it fails during the ninja build. Have you seen the following error before?

../core/conversion/conversion.h(5): fatal error C1083: Cannot open include file: 'NvInfer.h': No such file or directory

• I then tried downloading the artifact from the GHA job and installing, which worked.

Based on the GHA-installed package, I ran the CI-failing tests, for which I have the following results:

• test_bert_base_uncased for both export and compile fails with transformers==4.41.0, but passes with transformers==4.39.3. Could you try fixing this version in the .yml?
• test_linear_3_multi_dim_matrix + test_linear_4_multi_dim_matrix - these pass/fail nondeterministically for me on Windows. They fail more often and with larger variance than they do on Linux, but failures on this test have been seen on both systems. Needs further investigation on my end.
• test_bert_base_uncased in the Torchscript path causes a Windows fatal exception on my machine as well - this should be skipped on Windows only while it is investigated.
  • The above is also resolved by using transformers==4.39.3 on my machine

[narendasan]

LGTM, Thanks for all the hard work!