Test PR #1

bogdansalyp · 2025-02-13T21:53:15Z

Fix a bug in set float32 precision (Fix a bug in set float32 precision pytorch/torchtune#2271)
Construct EarlyFusion's encoder_token_ids on correct device (Construct EarlyFusion's encoder_token_ids on correct device pytorch/torchtune#2276)
Sample packing for ConcatDataset (Sample packing for ConcatDataset pytorch/torchtune#2278)
Added Distributed(Tensor Parallel) Inference Recipe (Added Distributed(Tensor Parallel) Inference Recipe pytorch/torchtune#2245)
Logging resolved config (Logging resolved config pytorch/torchtune#2274)
Removing SimPOLoss (Removing SimPOLoss pytorch/torchtune#2290)
Proper prefix handling in EarlyFusion sd hooks (Proper prefix handling in EarlyFusion sd hooks pytorch/torchtune#2291)
Remove deprecated components for 0.6.0 (Remove deprecated components for 0.6.0 pytorch/torchtune#2293)
Update the e2e flow tutorial to fix errors of generate (Update the e2e flow tutorial to fix errors of generate pytorch/torchtune#2251)
profiling ops on xpu (profiling ops on xpu pytorch/torchtune#2249)
Refactored modules/tokenizers to be a subdir of modules/transforms (Refactored modules/tokenizers to be a subdir of modules/transforms pytorch/torchtune#2231)
Update model builders (Update model builders pytorch/torchtune#2282)
[EZ] Only log deprecation warning on rank zero ([EZ] Only log deprecation warning on rank zero pytorch/torchtune#2308)
[ez] Add output_dir field to a couple configs ([ez] Add output_dir field to a couple configs pytorch/torchtune#2309)
Disable DSD and fix bitsandbytes test (Disable DSD and fix bitsandbytes test pytorch/torchtune#2314)
fix state dict hook for early fusion models (fix state dict hook for early fusion models pytorch/torchtune#2317)
Adding reverse and symmetric KLD losses (Adding reverse and symmetric KLD losses pytorch/torchtune#2094)
'tune cat' command for pretty printing configuration files ([WIP] 'tune cat' command for pretty printing configuration files pytorch/torchtune#2298)
Use checkout@v4 / upload@v4 for docs build (Use checkout@v4 / upload@v4 for docs build pytorch/torchtune#2322)
Fix stop tokens in PPO (Fix stop tokens in PPO pytorch/torchtune#2304)
Update PT pin to 2.6 for modules/_export (Update PT pin for modules/_export pytorch/torchtune#2336)
Update to proper EOS ids for Qwen2 and Qwen2.5 (Update to proper EOS ids for Qwen2 and Qwen2.5 pytorch/torchtune#2342)
Multinode support in torchtune (Multinode support in torchtune pytorch/torchtune#2301)
[Bug Fix]Disable DSD for saving ckpt ([Bug Fix]Disable DSD for saving ckpt pytorch/torchtune#2346)
Update README for multinode (Update README for multinode pytorch/torchtune#2348)
added tie_word_embeddings to llama3_2 models (added tie_word_embeddings to llama3_2 models pytorch/torchtune#2331)
Fix saving adapter weights after disabling DSD (Fix saving adapter weights after disabling DSD pytorch/torchtune#2351)
Remove "ft-" prefix from checkpoint shards. (Remove "ft-" prefix from checkpoint shards. pytorch/torchtune#2354)
Full DPO Distributed (Full DPO Distributed pytorch/torchtune#2275)
[Fix Test] Fix failed generation test by pining pytorch nightlies ([Fix Test] Fix failed generation test by pining pytorch nightlies pytorch/torchtune#2362)
TP + FSDP distributed training (full finetuning) (TP + FSDP distributed training (full finetuning) pytorch/torchtune#2330)
Add max-autotune try/except if flex attn breaks (Add max-autotune try/except if flex attn breaks pytorch/torchtune#2357)
readme updates for full DPO distributed recipe (readme updates for full DPO distributed recipe pytorch/torchtune#2363)
Add helper functions for pruning old checkpoints
Fix Qwen config (Fix Qwen config pytorch/torchtune#2377)
feat: Added cfg.cudnn_deterministic_mode flag (feat: Added cfg.cudnn_deterministic_mode flag pytorch/torchtune#2367)
Add Phi4 (Add Phi4 pytorch/torchtune#2197)
Add tests and implementation for disabling dropout layers in models (Add tests and implementation for disabling dropout layers in models pytorch/torchtune#2378)
Step based checkpointing and tests
Fix helper function tests
Silly linting rule
Allow kwargs in all checkpointers for BC with step
Add Phi4 to README.md (nit: Phi4 to readme pytorch/torchtune#2383)
Implements MLFlowLogger (Implements MLFlowLogger pytorch/torchtune#2365)
Stub
'ft-' prefix occurrence removal ('ft-' prefix occurrence removal pytorch/torchtune#2385)
check if log_dir is not none (check if log_dir is not none pytorch/torchtune#2389)
HF tokenizers: initial base tokenizer support (HF tokenizers: initial base tokenizer support pytorch/torchtune#2350)
Simplify README and prominently display recipes (Simplify README and prominently display recipes pytorch/torchtune#2349)
Renamed parallelize_plan to tensor_parallel_plan (Renamed parallelize_plan to tensor_parallel_plan pytorch/torchtune#2387)
Fix optimizer_in_backward at loading opt_state_dict in distributed recipes (Fix optimizer_in_backward at loading opt_state_dict in distributed recipes pytorch/torchtune#2390)
Resume from recipe state in step_x
Correct step for resume from checkpoint
fix: Fixed global step restoration from recipe_state

…2276)

Co-authored-by: JessicaZhong <[email protected]>

Co-authored-by: iseeyuan <[email protected]> Co-authored-by: RdoubleA <[email protected]>

…ytorch#2231)

Co-authored-by: JessicaZhong <[email protected]>

)

Co-authored-by: Joe Cummings <[email protected]>

Co-authored-by: JessicaZhong <[email protected]>

Co-authored-by: Salman Mohammadi <[email protected]> Co-authored-by: Bogdan Salyp <[email protected]>

…torch#2362) Co-authored-by: JessicaZhong <[email protected]>

Co-authored-by: JessicaZhong <[email protected]> Co-authored-by: joecummings <[email protected]>

Co-authored-by: Felipe Mello <[email protected]>

Co-authored-by: JessicaZhong <[email protected]>

Co-authored-by: Felipe Mello <[email protected]> Co-authored-by: Mark Obozov <[email protected]> Co-authored-by: ebsmothers <[email protected]>

…ytorch#2378)

Co-authored-by: Felipe Mello <[email protected]>

Co-authored-by: salman <[email protected]>

…cipes (pytorch#2390) Co-authored-by: Salman Mohammadi <[email protected]> Co-authored-by: Evan Smothers <[email protected]>

Nicorgi and others added 30 commits January 15, 2025 22:10

Fix a bug in set float32 precision (pytorch#2271)

890deab

Construct EarlyFusion's encoder_token_ids on correct device (pytorch#…

7747db1

…2276)

Sample packing for ConcatDataset (pytorch#2278)

1036095

Added Distributed(Tensor Parallel) Inference Recipe (pytorch#2245)

779569e

Co-authored-by: JessicaZhong <[email protected]>

Logging resolved config (pytorch#2274)

75965d4

Removing SimPOLoss (pytorch#2290)

9b50628

Proper prefix handling in EarlyFusion sd hooks (pytorch#2291)

d7afc40

Remove deprecated components for 0.6.0 (pytorch#2293)

b8790ce

Update the e2e flow tutorial to fix errors of generate (pytorch#2251)

90fd2d3

Co-authored-by: iseeyuan <[email protected]> Co-authored-by: RdoubleA <[email protected]>

profiling ops on xpu (pytorch#2249)

5764650

Refactored modules/tokenizers to be a subdir of modules/transforms (p…

3cceb86

…ytorch#2231)

Update model builders (pytorch#2282)

23896c3

[EZ] Only log deprecation warning on rank zero (pytorch#2308)

46d8153

[ez] Add output_dir field to a couple configs (pytorch#2309)

d4465c8

Disable DSD and fix bitsandbytes test (pytorch#2314)

d3b39cf

fix state dict hook for early fusion models (pytorch#2317)

be4ff50

Co-authored-by: JessicaZhong <[email protected]>

Adding reverse and symmetric KLD losses (pytorch#2094)

6764618

'tune cat' command for pretty printing configuration files (pytorch#2298

e6b9064

)

Use checkout@v4 / upload@v4 for docs build (pytorch#2322)

afd5bc4

Co-authored-by: Joe Cummings <[email protected]>

Fix stop tokens in PPO (pytorch#2304)

6487029

Update PT pin to 2.6 for modules/_export (pytorch#2336)

059cad9

Update to proper EOS ids for Qwen2 and Qwen2.5 (pytorch#2342)

9475b5a

Multinode support in torchtune (pytorch#2301)

4b364f2

[Bug Fix]Disable DSD for saving ckpt (pytorch#2346)

b7432df

Co-authored-by: JessicaZhong <[email protected]>

Update README for multinode (pytorch#2348)

9f14fe9

added tie_word_embeddings to llama3_2 models (pytorch#2331)

a226a58

Fix saving adapter weights after disabling DSD (pytorch#2351)

a8aeb13

Co-authored-by: JessicaZhong <[email protected]>

Remove "ft-" prefix from checkpoint shards. (pytorch#2354)

9b38360

Full DPO Distributed (pytorch#2275)

fb52557

Co-authored-by: Salman Mohammadi <[email protected]> Co-authored-by: Bogdan Salyp <[email protected]>

[Fix Test] Fix failed generation test by pining pytorch nightlies (py…

8c9235e

…torch#2362) Co-authored-by: JessicaZhong <[email protected]>

acisseJZhong and others added 25 commits February 9, 2025 19:19

TP + FSDP distributed training (full finetuning) (pytorch#2330)

9da35c7

Co-authored-by: JessicaZhong <[email protected]> Co-authored-by: joecummings <[email protected]>

Add max-autotune try/except if flex attn breaks (pytorch#2357)

57f7cc1

Co-authored-by: Felipe Mello <[email protected]>

readme updates for full DPO distributed recipe (pytorch#2363)

b3964af

Add helper functions for pruning old checkpoints

c5093b7

Fix Qwen config (pytorch#2377)

a0f74c3

Co-authored-by: JessicaZhong <[email protected]>

feat: Added cfg.cudnn_deterministic_mode flag (pytorch#2367)

386ca8d

Add Phi4 (pytorch#2197)

c09f9b6

Co-authored-by: Felipe Mello <[email protected]> Co-authored-by: Mark Obozov <[email protected]> Co-authored-by: ebsmothers <[email protected]>

Add tests and implementation for disabling dropout layers in models (p…

b9e5b1e

…ytorch#2378)

Step based checkpointing and tests

b0648fd

Fix helper function tests

46f7c67

Silly linting rule

4385bff

Allow kwargs in all checkpointers for BC with step

d39f3a3

Add Phi4 to README.md (pytorch#2383)

7f3e70e

Implements MLFlowLogger (pytorch#2365)

a965fb0

Stub

2b16947

'ft-' prefix occurrence removal (pytorch#2385)

d5d85c5

check if log_dir is not none (pytorch#2389)

0792bcf

Co-authored-by: Felipe Mello <[email protected]>

HF tokenizers: initial base tokenizer support (pytorch#2350)

e1386d1

Simplify README and prominently display recipes (pytorch#2349)

f67ccda

Co-authored-by: salman <[email protected]>

Renamed parallelize_plan to tensor_parallel_plan (pytorch#2387)

cb83655

Fix optimizer_in_backward at loading opt_state_dict in distributed re…

2128559

…cipes (pytorch#2390) Co-authored-by: Salman Mohammadi <[email protected]> Co-authored-by: Evan Smothers <[email protected]>

Merge remote-tracking branch 'upstream/main' into impl-step-based-ckpt

374ffa7

Resume from recipe state in step_x

8124fc7

Correct step for resume from checkpoint

316d655

fix: Fixed global step restoration from recipe_state

dde1c34

bogdansalyp closed this Feb 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test PR #1

Test PR #1

bogdansalyp commented Feb 13, 2025

Test PR #1

Test PR #1

Conversation

bogdansalyp commented Feb 13, 2025