support GPU inference in torchscript model for v2.5 / v2.6

[taoleicn]

This PR works for master branch, v2.5 and v2.6 release

A non-trivial PR to support GPU inference in torchscript

• Load CUDA kernels as non-python modules; this is needed for torchscript compilation
• Refactored CUDA APIs as functions that return output as tensors, instead of procedures that modify some passed-in tensors.
• Added a workaround in case TS tries to locate and compile CUDA methods on machines that don't have CUDA / GPUs
• The refactored code has passed the forward() & backward() test.
• I also checked the outputs are the same for the non-torchscript and torchscript versions of the same model.

[hpasapp]

oh is this 2.x? for some reason I assumed it was 3.x. Though it does say 2.x in the title actually, now that I look ... :)

[hpasapp]

oh, there are two PRs :P

[hpasapp]

What pytest tests do we have for the code affected by these changes [edit: or at least, an automated test script, such as bash, that I could run by hand]? I understand we have to run such tests by hand, since CCIE doesnt do that for us.

[taoleicn]

What pytest tests do we have for the code affected by these changes [edit: or at least, an automated test script, such as bash, that I could run by hand]? I understand we have to run such tests by hand, since CCIE doesnt do that for us.

@hpasapp The existing unit tests should cover most of the code paths for non-cuda usage. I also modified the C++ test to ensure the changes work for C++ and for CPU inference.

[taoleicn]

Also @cdfox-asapp feels we don't have to merge this PR into master, since we need to merge 3.0.0 dev into master anyway. The changes are identical to the PR for v3.0.0. I can make a release on this branch for Chris's project.
What do you think? @hpasapp

[hpasapp]

Advantage of not merging this branch into master? (not merging it means an additional branch to maintain and reason about.)

[taoleicn]

🎉 thanks @hpasapp @cdfox-asapp !