Code

PEFT support for Online DPO#2041

Code

Merged

qgallouedec merged 31 commits intomainhuggingface/trl:mainfrom kaisthuggingface/trl:kaistCopy head branch name to clipboardSep 13, 2024

+209-46

Commits

Commits on Sep 4, 2024

• Promote PPOv2Trainer and PPOv2Config to top-level import

  qgallouedec

  committed
  96ae02a

  Copy full SHA for 96ae02a

• Deprecate PPOTrainer and PPOConfig

  qgallouedec

  committed
  65990de

  Copy full SHA for 65990de

Commits on Sep 5, 2024

• changes

  qgallouedec

  committed
  74e5302

  Copy full SHA for 74e5302

Commits on Sep 9, 2024

• Revert "Promote PPOv2Trainer and PPOv2Config to top-level import"

  Show description for 3739e8d
  qgallouedec

  committed
  3739e8d

  Copy full SHA for 3739e8d

• Revert "Deprecate PPOTrainer and PPOConfig"

  Show description for 34dc5b8
  qgallouedec

  committed
  34dc5b8

  Copy full SHA for 34dc5b8

• Merge branch 'main' into kaist

  qgallouedec

  authored
  4412f35

  Copy full SHA for 4412f35

• Merge branch 'main' into kaist

  qgallouedec

  authored
  58b16db

  Copy full SHA for 58b16db

Commits on Sep 10, 2024

• peft

  qgallouedec

  committed
  8ee7b0c

  Copy full SHA for 8ee7b0c

• peft

  qgallouedec

  committed
  cff0ddc

  Copy full SHA for cff0ddc

Commits on Sep 12, 2024

• try to simplify

  qgallouedec

  committed
  60a45fc

  Copy full SHA for 60a45fc

• revert utils changes

  qgallouedec

  committed
  72b1e89

  Copy full SHA for 72b1e89

• Merge branch 'main' into kaist

  qgallouedec

  authored
  7842c34

  Copy full SHA for 7842c34

• update dpo script

  qgallouedec

  committed
  9f5b9f8

  Copy full SHA for 9f5b9f8

Commits on Sep 13, 2024

• peft

  qgallouedec

  committed
  7926092

  Copy full SHA for 7926092

• style

  qgallouedec

  committed
  cfcaa74

  Copy full SHA for cfcaa74

• revert gitignore

  qgallouedec

  committed
  12f909e

  Copy full SHA for 12f909e

• test_online_dpo_peft

  qgallouedec

  committed
  f930112

  Copy full SHA for f930112

• ref model

  qgallouedec

  committed
  ff148b0

  Copy full SHA for ff148b0

• peft example command

  qgallouedec

  committed
  992805d

  Copy full SHA for 992805d

• typo

  qgallouedec

  committed
  e9b07bd

  Copy full SHA for e9b07bd

• remove param.requires_grad = False for the reward model

  qgallouedec

  committed
  cf39657

  Copy full SHA for cf39657

• make model required arg

  qgallouedec

  committed
  c66414f

  Copy full SHA for c66414f

• update example script

  qgallouedec

  committed
  3928009

  Copy full SHA for 3928009

• update xpo trainer

  qgallouedec

  committed
  b15ddea

  Copy full SHA for b15ddea

• Update examples/scripts/dpo_online.py

  Show description for 3c85b93
  

  qgallouedec

  and

  lewtun

  authored
  3c85b93

  Copy full SHA for 3c85b93

• Update examples/scripts/dpo_online.py

  Show description for 4a38809
  

  qgallouedec

  and

  lewtun

  authored
  4a38809

  Copy full SHA for 4a38809

• Merge branch 'main' into kaist

  qgallouedec

  authored
  a30589c

  Copy full SHA for a30589c

• Merge branch 'main' into kaist

  qgallouedec

  authored
  37c7fca

  Copy full SHA for 37c7fca

• Merge branch 'main' into kaist

  qgallouedec

  authored
  44c1750

  Copy full SHA for 44c1750

• Merge branch 'main' into kaist

  qgallouedec

  authored
  2a61af1

  Copy full SHA for 2a61af1

• merge and unload

  qgallouedec

  committed
  0246c5f

  Copy full SHA for 0246c5f