Code

[FlashInfer] Upgrade to 0.2.0#11194

Code

Merged

youkaichao merged 55 commits intovllm-project:mainvllm-project/vllm:mainfrom abmfy:flashinfer-0.2abmfy/vllm-flashinfer:flashinfer-0.2Copy head branch name to clipboardJan 27, 2025

+257-75

Commits

Commits on Dec 14, 2024

• [misc] remove deprecated call to end_forward in flashinfer backend

  Show description for 269f965
  abmfy

  committed
  269f965

  Copy full SHA for 269f965

Commits on Dec 20, 2024

• [flashinfer] upgrade to flashinfer 0.2.0

  Show description for 8c375a3
  abmfy

  committed
  8c375a3

  Copy full SHA for 8c375a3

• [style] fix yapf check

  Show description for a62b854
  abmfy

  committed
  a62b854

  Copy full SHA for a62b854

Commits on Dec 31, 2024

• [FlashInfer] Pass infered global hyperparameters to plan

  Show description for b37ff55
  abmfy

  committed
  b37ff55

  Copy full SHA for b37ff55

• [FlashInfer] Cache inferred global hyperparameters

  Show description for 72bdf7e
  abmfy

  committed
  72bdf7e

  Copy full SHA for 72bdf7e

• [Misc] Use typing.Optional for Python 3.9 compatability

  Show description for 97dcedc
  abmfy

  committed
  97dcedc

  Copy full SHA for 97dcedc

• [Style] Fix lint errors

  Show description for 56798c5
  abmfy

  committed
  56798c5

  Copy full SHA for 56798c5

Commits on Jan 22, 2025

• Merge branch 'main' into flashinfer-0.2

  Show description for 706a6f6
  abmfy

  committed
  706a6f6

  Copy full SHA for 706a6f6

• [FlashInfer] Cache global hyperparameters in AttentionMetadataBuilder instance

  Show description for dacb6af
  abmfy

  committed
  dacb6af

  Copy full SHA for dacb6af

• [Style] Fix ruff

  Show description for 06fa7cc
  abmfy

  committed
  06fa7cc

  Copy full SHA for 06fa7cc

Commits on Jan 23, 2025

• [FlashInfer] Get per layer params from vllm config

  Show description for bc480b0
  abmfy

  committed
  bc480b0

  Copy full SHA for bc480b0

Commits on Jan 24, 2025

• [FlashInfer] Store vllm config in attention state

  Show description for 5a70aac
  abmfy

  committed
  5a70aac

  Copy full SHA for 5a70aac

• [CI] Update FlashInfer version

  Show description for e0397e9
  abmfy

  committed
  e0397e9

  Copy full SHA for e0397e9

• format

  Show description for ec49257
  

  youkaichao

  authored and

  abmfy

  committed
  ec49257

  Copy full SHA for ec49257

• Merge branch 'main' into flashinfer-0.2

  Show description for 500ff5b
  abmfy

  committed
  500ff5b

  Copy full SHA for 500ff5b

• [Misc] Add space in assert message

  Show description for bde6807
  abmfy

  committed
  bde6807

  Copy full SHA for bde6807

• [FlashInfer] Warn on models with interleaved attention

  Show description for 69d7c8d
  abmfy

  committed
  69d7c8d

  Copy full SHA for 69d7c8d

• [Test] Change backend to flash_attn for gemma in compile tests

  Show description for d4d63dc
  abmfy

  committed
  d4d63dc

  Copy full SHA for d4d63dc

Commits on Jan 25, 2025

• fix inconsistent vllm config

  Show description for 6e7e933
  youkaichao

  committed
  6e7e933

  Copy full SHA for 6e7e933

• Merge branch 'flashinfer-0.2' of github.com:abmfy/vllm-flashinfer into flashinfer-0.2

  Show description for 0b47067
  abmfy

  committed
  0b47067

  Copy full SHA for 0b47067

• [Test] Skip tests for Gemma2 with FlashInfer backend

  Show description for f6e33a7
  abmfy

  committed
  f6e33a7

  Copy full SHA for f6e33a7

• [CI] Build FlashInfer from source

  Show description for 847a4d6
  abmfy

  committed
  847a4d6

  Copy full SHA for 847a4d6

• [CI] Fix FlashInfer build command

  Show description for 5b0fe64
  abmfy

  committed
  5b0fe64

  Copy full SHA for 5b0fe64

• [CI] Fix Dockerfile

  Show description for 69445cd
  abmfy

  committed
  69445cd

  Copy full SHA for 69445cd

• [CI] Fix FlashInfer AOT build in Dockerfile

  Show description for 963aff7
  abmfy

  committed
  963aff7

  Copy full SHA for 963aff7

Commits on Jan 26, 2025

• fix flashinfer docker build

  Show description for ae9da66
  youkaichao

  committed
  ae9da66

  Copy full SHA for ae9da66

• Merge branch 'main' into flashinfer-0.2

  youkaichao

  committed
  afa377c

  Copy full SHA for afa377c

• fix build command

  Show description for 269e1eb
  youkaichao

  committed
  269e1eb

  Copy full SHA for 269e1eb

• move command

  Show description for 2e50ab8
  youkaichao

  committed
  2e50ab8

  Copy full SHA for 2e50ab8

• unify to use setup.py

  Show description for 0fe979d
  youkaichao

  committed
  0fe979d

  Copy full SHA for 0fe979d

• fix cd

  Show description for 3dd209c
  youkaichao

  committed
  3dd209c

  Copy full SHA for 3dd209c

• fix recursive clone

  Show description for bcd04fd
  youkaichao

  committed
  bcd04fd

  Copy full SHA for bcd04fd

• comment

  Show description for bb44221
  youkaichao

  committed
  bb44221

  Copy full SHA for bb44221

• [CI] Use precompiled FlashInfer AOT wheel

  Show description for 5ca67ae
  abmfy

  committed
  5ca67ae

  Copy full SHA for 5ca67ae

• [CI] Temporarily switch to CUDA develop image for vllm-base

  Show description for 3c89bfb
  abmfy

  committed
  3c89bfb

  Copy full SHA for 3c89bfb

• Merge branch 'main' into flashinfer-0.2

  Show description for 293fdd6
  abmfy

  committed
  293fdd6

  Copy full SHA for 293fdd6

• also install jit build dependency

  Show description for 5d8ad22
  youkaichao

  committed
  5d8ad22

  Copy full SHA for 5d8ad22

• [FlashInfer] Fix type of k_scale and v_scale

  Show description for 4d57ef9
  abmfy

  committed
  4d57ef9

  Copy full SHA for 4d57ef9

• Merge branch 'main' into flashinfer-0.2

  Show description for 33ff07b
  abmfy

  committed
  33ff07b

  Copy full SHA for 33ff07b

• Merge branch 'flashinfer-0.2' of github.com:abmfy/vllm-flashinfer into flashinfer-0.2

  Show description for ef15977
  abmfy

  committed
  ef15977

  Copy full SHA for ef15977

Commits on Jan 27, 2025

• fix reshape_and_cache_flash

  Show description for 21efc67
  youkaichao

  committed
  21efc67

  Copy full SHA for 21efc67

• use new flashinfer

  Show description for a6b6fe8
  youkaichao

  committed
  a6b6fe8

  Copy full SHA for a6b6fe8

• Merge branch 'main' into flashinfer-0.2

  youkaichao

  committed
  1f13235

  Copy full SHA for 1f13235

• update v1 tests

  Show description for f17dbc3
  youkaichao

  committed
  f17dbc3

  Copy full SHA for f17dbc3

• refactor test

  Show description for 506b641
  youkaichao

  committed
  506b641

  Copy full SHA for 506b641

• revert

  Show description for 2e476a2
  youkaichao

  committed
  2e476a2

  Copy full SHA for 2e476a2

• add comments

  Show description for 95b5493
  youkaichao

  committed
  95b5493

  Copy full SHA for 95b5493

• only check compile when loading

  Show description for 55b55d3
  youkaichao

  committed
  55b55d3

  Copy full SHA for 55b55d3

• test in ci?

  Show description for 1f80aee
  youkaichao

  committed
  1f80aee

  Copy full SHA for 1f80aee

• fix one test

  Show description for 5be3783
  youkaichao

  committed
  5be3783

  Copy full SHA for 5be3783

• fix test_flashinfer_prefill_with_paged_kv

  Show description for 071a68e
  youkaichao

  committed
  071a68e

  Copy full SHA for 071a68e

• relax test for prefill

  Show description for 0e0f57f
  youkaichao

  committed
  0e0f57f

  Copy full SHA for 0e0f57f

• fix test_flashinfer_prefill_with_paged_fp8_kv

  Show description for 2134e77
  youkaichao

  committed
  2134e77

  Copy full SHA for 2134e77

• relax test for prefill

  Show description for 8e42297
  youkaichao

  committed
  8e42297

  Copy full SHA for 8e42297

• fix test_flashinfer_decode_with_paged_fp8_kv

  Show description for b4a7992
  youkaichao

  committed
  b4a7992

  Copy full SHA for b4a7992