Is Multiple-GPU support available now?

[demongolem-biz]

Is your feature/enhancement request related to a problem? Please describe.
I have 2 GPUs. Only 1 is being accessed. With that one, I get a CUDA out of memory error. I am hoping that by leveraging both GPUs this situation will improve.

Describe the solution you'd like
Allow more than 1 gpu to be specified.

Additional context
#389 was closed as duplicate of #37. But, #37 was closed without addressing #389 and by giving some magic hand-waving that GPU support is better now (circa 2020).

[ayan-iiitd]

An update on this will be really appreciated. I am trying to run inference on a dataset with more than 300k entries. For one instance it takes about 1.05 secs and uses about 10% of one of two P5000s. Please give an update on this.

[rajharshiitb]

Yeah, any update on this would be really helpful.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[fernando080]

Any updates about this issue? ^^

[alanakbik]

There is a PR for Lightning Lite integration #2700 that will add this feature and should be merged soon-ish!