Code

Support per-request seed#2514

Code

Merged

simon-mo merged 12 commits intovllm-project:mainvllm-project/vllm:mainfrom njhill:seednjhill/vllm:seedCopy head branch name to clipboardFeb 21, 2024

+289-84

Commits

Commits on Feb 21, 2024

• Support per-request seed

  njhill

  committed
  824db68

  Copy full SHA for 824db68

• Fix setting generators

  njhill

  committed
  ec7c3bf

  Copy full SHA for ec7c3bf

• Free per-request generator state upon completion

  njhill

  committed
  3b71f4b

  Copy full SHA for 3b71f4b

• Add parameter verification and tests

  njhill

  committed
  08fb671

  Copy full SHA for 08fb671

• Fixes and simplification

  Show description for 8ec7ced
  njhill

  committed
  8ec7ced

  Copy full SHA for 8ec7ced

• Stash generator in seq-group-specific dict

  Show description for 1cae401
  njhill

  committed
  1cae401

  Copy full SHA for 1cae401

• Remove _generator field from SamplingParams

  njhill

  committed
  5db801e

  Copy full SHA for 5db801e

• Add tests for deterministic output

  njhill

  committed
  f264ab2

  Copy full SHA for f264ab2

• Use dataclass instead of dict for sequence group state

  Show description for 364cdd2
  njhill

  committed
  364cdd2

  Copy full SHA for 364cdd2

• Improve test_sampler_mixed to test seed determinism

  njhill

  committed
  a0eebef

  Copy full SHA for a0eebef

• Change seeded generate test to use mixed batch

  njhill

  committed
  1a774fd

  Copy full SHA for 1a774fd

• Remove no-longer-applicable comment

  njhill

  committed
  57d9536

  Copy full SHA for 57d9536