Support different AMP & buffer configurations in one experiment, fix minor bugs#389

Merged

justheuristic merged 20 commits intomasterfrom fp16

Sep 28, 2021

+131-22

Commits on Sep 25, 2021

Commits on Sep 27, 2021

Update hivemind/optim/grad_scaler.py

justheuristic
and
borzunov
authored
Update hivemind/optim/grad_scaler.py

justheuristic
and
borzunov
authored
Update hivemind/optim/collaborative.py

justheuristic
and
borzunov
authored
Update hivemind/optim/collaborative.py

justheuristic
and
borzunov
authored
Update hivemind/optim/collaborative.py

justheuristic
and
borzunov
authored
Update hivemind/optim/grad_scaler.py

justheuristic
and
borzunov
authored
Update hivemind/optim/grad_scaler.py

justheuristic
and
borzunov
authored
explain why allow_fp16

TimDettmers
authored and
justheuristic
committed
Merge remote-tracking branch 'origin/fp16' into fp16
justheuristic
committed
explain why allow_fp16
justheuristic
committed
Update hivemind/optim/grad_scaler.py

justheuristic
and
borzunov
authored
Update hivemind/optim/collaborative.py

justheuristic
and
borzunov
authored
explain why allow_fp16
justheuristic
committed
extended docstr
justheuristic
committed
review

TimDettmers
authored and
justheuristic
committed
review

borzunov
authored and
justheuristic
committed