-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
1/n Call training_type_plugin
collective functions directly instead of going through the Accelerator
#9677
Conversation
4e833e2
to
eacdbf2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
training type plugin is already a top-level property on the trainer: https://github.com/PyTorchLightning/pytorch-lightning/blob/41e3be197f5a2fd0f65b37b743ebfd157a55595d/pytorch_lightning/trainer/trainer.py#L1445-L1447
as this is part of #9426 , we should directly update callsites to call collectives via the training type plugin.
do you want to mark the collectives defined on the accelerator as deprecated in another PR? same question for LightningDistributed
eacdbf2
to
8ffdb29
Compare
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
Codecov Report
@@ Coverage Diff @@
## master #9677 +/- ##
=======================================
- Coverage 93% 89% -4%
=======================================
Files 179 179
Lines 15314 15311 -3
=======================================
- Hits 14208 13591 -617
- Misses 1106 1720 +614 |
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Adrian Wälchli <[email protected]>
for more information, see https://pre-commit.ci
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
Co-authored-by: ananthsub <[email protected]>
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
f8a7483
to
ed53c0d
Compare
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
d3f200b
to
abf0833
Compare
for more information, see https://pre-commit.ci
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
pytorch_lightning/plugins/training_type/training_type_plugin.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Rohit Gupta <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM !
training_type_plugin
collective functions directly instead of going through the Accelerator
What does this PR do?
1/n for Directly call TrainingTypePlugin APIs instead of going through the Accelerator
Take incremental step for collective consolidation
Call training type plugin collective function directly instead of go through accelerator
2/n Consolidate collective functions - collective base and subclasses (#9414) will take time to polish the API, so have this implementation to unblock precision plugin refactor
Part of #7534
Part of #9426
Does your PR introduce any breaking changes? If yes, please list them.
Before submitting
PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:
Did you have fun?
Make sure you had fun coding 🙃