Code

Grad accumulation and memory bugfix#220

Code

Merged

edbeeching merged 12 commits intomainhuggingface/trl:mainfrom grad-accu-memory-bugfixhuggingface/trl:grad-accu-memory-bugfixCopy head branch name to clipboardMar 16, 2023

+152-37

Commits

Commits on Mar 15, 2023

• adds args and grad accum steps to sentiment examples

  edbeeching

  committed
  bdab2f1

  Copy full SHA for bdab2f1

• updates to minibatch size in peft 20b example

  edbeeching

  committed
  9cc5222

  Copy full SHA for 9cc5222

• adds arg and grad acc to toxicity example

  edbeeching

  committed
  2e603e2

  Copy full SHA for 2e603e2

• adds detach to all entries in the step stats to reduce memory usage

  edbeeching

  committed
  c028e6a

  Copy full SHA for c028e6a

• style

  edbeeching

  committed
  16995de

  Copy full SHA for 16995de

• style2

  edbeeching

  committed
  4939869

  Copy full SHA for 4939869

• adds accelerator accumulation context

  edbeeching

  committed
  543432c

  Copy full SHA for 543432c

• style

  edbeeching

  committed
  a17d49a

  Copy full SHA for a17d49a

• makes gradient_accumulation_steps part of the PPOConfig

  edbeeching

  committed
  cf2db12

  Copy full SHA for cf2db12

• style

  edbeeching

  committed
  89c02aa

  Copy full SHA for 89c02aa

• Update trl/trainer/ppo_trainer.py

  Show description for c7d0731
  

  edbeeching

  and

  younesbelkada

  authored
  c7d0731

  Copy full SHA for c7d0731

Commits on Mar 16, 2023

• style

  edbeeching

  committed
  6cef869

  Copy full SHA for 6cef869