[Fix] Move init dist connection into the setup function#6506

Merged

SeanNaren merged 33 commits intomasterfrom fix/setup_ddp_hook

Mar 18, 2021

+139-100

Commits on Mar 13, 2021

Commits on Mar 14, 2021

Add missing mock, ensure custom plugin does not create children process
SeanNaren
committed
Merge branch 'master' into fix/setup_ddp_hook
SeanNaren
committed

Commits on Mar 15, 2021

Merge branch 'master' into fix/setup_ddp_hook
SeanNaren
authored
Skip test on windows
SeanNaren
committed
Update deepspeed to init connection in setup
SeanNaren
committed
Do not initialize distributed module
SeanNaren
committed

Commits on Mar 16, 2021

Move DeepSpeed tests to special tests since dist communication is being set up
SeanNaren
committed
Merge branch 'master' into fix/setup_ddp_hook
SeanNaren
committed
Special the test to see if this fixes CI
SeanNaren
committed
Delete accelerator connector test to see if its causing build to fail
SeanNaren
committed
Delete deepspeed test
SeanNaren
committed
Revert "Delete accelerator connector test to see if its causing build to fail"
SeanNaren
committed
Revert "Delete deepspeed test"
SeanNaren
committed
Reverse hook
SeanNaren
committed
Reverse setup hooks to debug again
SeanNaren
committed

Commits on Mar 17, 2021

Add todo so i know where i left off
SeanNaren
committed
For single device move in pre_dispatch after setup function
SeanNaren
committed
Merge branch 'master' into fix/setup_ddp_hook
SeanNaren
committed
Add additional model to device hook if any additional parameters have been set
SeanNaren
committed
See if we can enable deepspeed tests
SeanNaren
committed
Revert "See if we can enable deepspeed tests"
SeanNaren
committed

Commits on Mar 18, 2021

See if this hook approach works
SeanNaren
committed
Introduce new granular hooks
SeanNaren
committed
Remove import, fix tpu spawn by moving the function to setup
SeanNaren
committed
Added missing special test
SeanNaren
committed
Merge branch 'master' into fix/setup_ddp_hook
SeanNaren
committed
Clean up the setup comment, since its run on train and test
SeanNaren
committed