Code

Add RoPE Interpolation#3564

Code

Merged

andreaskoepf merged 12 commits intoLAION-AI:mainLAION-AI/Open-Assistant:mainfrom shahules786:dev-falconshahules786/Open-Assistant:dev-falconCopy head branch name to clipboardJul 12, 2023

+274-3

Commits

Commits on Jul 11, 2023

• NTK scaled rope

  shahules786

  committed
  1cdb388

  Copy full SHA for 1cdb388

• added patching for falcon

  shahules786

  committed
  53189e4

  Copy full SHA for 53189e4

• fix for falcon layers

  shahules786

  committed
  252e96b

  Copy full SHA for 252e96b

• add rope scaling

  shahules786

  committed
  ef45c26

  Copy full SHA for ef45c26

• add dynamic ntk

  shahules786

  committed
  44b9522

  Copy full SHA for 44b9522

• rename

  shahules786

  committed
  b40c93d

  Copy full SHA for b40c93d

• added rope

  shahules786

  committed
  e5620b3

  Copy full SHA for e5620b3

• added sample config

  shahules786

  committed
  d0e9587

  Copy full SHA for d0e9587

• pre-commit

  shahules786

  committed
  522594e

  Copy full SHA for 522594e

• rmv changes

  shahules786

  committed
  cc14e5a

  Copy full SHA for cc14e5a

Commits on Jul 12, 2023

• cleanup

  shahules786

  committed
  eaae56c

  Copy full SHA for eaae56c

• added docs

  shahules786

  committed
  ef03d9b

  Copy full SHA for ef03d9b