Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FP8 + FSDP2 + torch.compile examples for PyTorch Lightning and Fabric #20440

Merged
merged 13 commits into from
Nov 26, 2024
Prev Previous commit
Next Next commit
Fix link
  • Loading branch information
lantiga committed Nov 25, 2024
commit 06c6af62cb1e0b356ab98d1f0262bf11fe3ec2b5
2 changes: 1 addition & 1 deletion docs/source-fabric/advanced/compile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ Here is an example:
The advantage here is that `parallelize` is called when sharding the model,
so :func:`torch.compile` is guaranteed to run on model shards and capture distributed operations.

Also, when using other libraries like `torch ao <https://github.com/pytorch/ao>`
Also, when using other libraries like `torch ao <https://github.com/pytorch/ao>`_
that need to be applied in a similar fashion, it's easy to reason about the sequence of calls
needed to achieve the equivalent of `compile(distributed(quantized(model)))`:

Expand Down
2 changes: 1 addition & 1 deletion docs/source-pytorch/advanced/compile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ Here is an example:
The advantage here is that `configure_model` is called when sharding the model,
so :func:`torch.compile` is guaranteed to run on model shards and capture distributed operations.

Also, when using other libraries like `torch ao <https://github.com/pytorch/ao>`
Also, when using other libraries like `torch ao <https://github.com/pytorch/ao>`_
that need to be applied in a similar fashion, it's easy to reason about the sequence of calls
needed to achieve the equivalent of `compile(distributed(quantized(model)))`:

Expand Down
Loading