Code

Move sampler into CUDA graph#1201

Code

Merged

merrymercy merged 14 commits intomainsgl-project/sglang:mainfrom sampler-cudagraphsgl-project/sglang:sampler-cudagraphCopy head branch name to clipboardAug 26, 2024

+336-110

Commits

Commits on Aug 24, 2024

• remove batch.sample

  hnyls2002

  committed
  017fa81

  Copy full SHA for 017fa81

• get penalty tensors

  hnyls2002

  committed
  d87b8db

  Copy full SHA for d87b8db

Commits on Aug 25, 2024

• move flashinfer sampling check out of sampler

  hnyls2002

  committed
  13c133d

  Copy full SHA for 13c133d

• flag: need_min_p_sampling

  hnyls2002

  committed
  e8cd731

  Copy full SHA for e8cd731

• compatible with CUDA graph

  hnyls2002

  committed
  269db62

  Copy full SHA for 269db62

• compatible with torch compile

  hnyls2002

  committed
  02a71d5

  Copy full SHA for 02a71d5

Commits on Aug 26, 2024

• Merge branch 'main' into sampler-cudagraph

  hnyls2002

  committed
  14c3fee

  Copy full SHA for 14c3fee

• update all models

  hnyls2002

  committed
  24fb28e

  Copy full SHA for 24fb28e

• Merge branch 'main' into sampler-cudagraph

  hnyls2002

  committed
  69d1089

  Copy full SHA for 69d1089

• Merge branch 'main' into sampler-cudagraph

  hnyls2002

  committed
  79acb6f

  Copy full SHA for 79acb6f

• reduce size of CUDA graph

  hnyls2002

  committed
  50bc6a1

  Copy full SHA for 50bc6a1

• fix

  hnyls2002

  committed
  a05bb67

  Copy full SHA for a05bb67

• fix torch compile

  hnyls2002

  committed
  7378d09

  Copy full SHA for 7378d09

• Merge branch 'main' into sampler-cudagraph

  zhyncs

  authored
  f71f0f9

  Copy full SHA for f71f0f9