Skip to content

Commit

Permalink
Distributed optimizer infrastructure for FP8 parameters (#1723)
Browse files Browse the repository at this point in the history 
* Add distopt support for param syncs with non-floating-point dtypes

Signed-off-by: Tim Moon <[email protected]>

* Update apex/contrib/optimizers/distributed_fused_adam.py

Co-authored-by: Masaki Kozuki <[email protected]>

---------

Signed-off-by: Tim Moon <[email protected]>
Co-authored-by: Masaki Kozuki <[email protected]>
  • Loading branch information
@timmoon10 @crcrpar
timmoon10 and crcrpar authored Sep 29, 2023
1 parent 6ff4548 commit 2386a91
Show file tree
Hide file tree
Showing 2 changed files with 280 additions and 83 deletions.
Loading

0 comments on commit 2386a91

Please sign in to comment.