Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Upstream sync 2024 04 12 #183

Merged
merged 44 commits into from
Apr 15, 2024
Merged

Upstream sync 2024 04 12 #183

merged 44 commits into from
Apr 15, 2024

Conversation

andy-neuma
Copy link
Member

@andy-neuma andy-neuma commented Apr 12, 2024 •
edited
Loading

SUMMARY:

  • upstream sync from "vllm" using cherry-pick. range b4543c8..7fd3949a0b1c6cd0dcd7066aca48d9d589f2f68e

NOTES:
vllm current 7fd3949 ... 7fd3949a0b1c6cd0dcd7066aca48d9d589f2f68e
vllm from last upstream sync b4543c8 ... b4543c8f6bf67a7f1a0d6d0fd6cf5697c7eeaabb

COMMAND:

git cherry-pick b4543c8..7fd3949a0b1c6cd0dcd7066aca48d9d589f2f68e

TEST PLAN:
runs on remote push and will trigger a NIGHTLY

egortolmachev and others added 30 commits April 12, 2024 09:03
@egortolmachev @t3ga
[Bugfix] Added Command-R GPTQ support (vllm-project#3849)
209c356 
Co-authored-by: Egor Tolmachev <[email protected]>
@Ki6an @esmeetu
[Bugfix] Enable Proper attention_bias Usage in Llama Model Configur…
b4b4e33 
…ation (vllm-project#3767)

Co-authored-by: roy <[email protected]>
@mawong-amd
[Hotfix][CI/Build][Kernel] CUDA 11.8 does not support layernorm optim…
ce01e55 
…izations (vllm-project#3782)
@esmeetu
[BugFix][Model] Fix commandr RoPE max_position_embeddings (vllm-proje…
88385bb 
…ct#3919)
@youkaichao
[Core] separate distributed_init from worker (vllm-project#3904)
f7db9ea
@cadedaniel
[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enabl…
7e06ab2 
…e hardware-agnostic speculative decoding" (vllm-project#3837)
@jsato8094
[Bugfix] Fix KeyError on loading GPT-NeoX (vllm-project#3925)
ddf95b9
@jpvillam-amd @gshtras @WoosukKwon
[ROCm][Hardware][AMD] Use Triton Kernel for default FA on ROCm (vllm-…
b2134e5 
…project#3643)

Co-authored-by: jpvillam <[email protected]>
Co-authored-by: Gregory Shtrasberg <[email protected]>
Co-authored-by: Woosuk Kwon <[email protected]>
@jeejeelee
[Misc] Avoid loading incorrect LoRA config (vllm-project#3777)
739c063
@zedong-peng
[Benchmark] Add cpu options to bench scripts (vllm-project#3915)
733ab62
@zhaotyer
[Bugfix] fix utils.py/merge_dict func TypeError: 'type' object is not…
cea1ec9 
… subscriptable (vllm-project#3955)

Co-authored-by: tianyi_zhao <[email protected]>
@huyiwen
[Bugfix] Fix logits processor when prompt_logprobs is not None (vllm-…
96da965 
…project#3899)
@tjohnson31415
[Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (vllm-pr…
ae0a64d 
…oject#3876)

Signed-off-by: Travis Johnson <[email protected]>
@WoosukKwon
[Bugfix][ROCm] Add numba to Dockerfile.rocm (vllm-project#3962)
a832b0f
@jamestwhedbee
[Model][AMD] ROCm support for 256 head dims for Gemma (vllm-project#3972
9622ae4 
)
@youkaichao @ywang96
[Doc] Add doc to state our model support policy (vllm-project#3948)
3389870 
Co-authored-by: Roger Wang <[email protected]>
@dmarasco
[Bugfix] Remove key sorting for guided_json parameter in OpenAi com…
d9e723e 
…patible Server (vllm-project#3945)
@fpaupier
[Doc] Fix getting stared to use publicly available model (vllm-projec…
71cbad1 
…t#3963)
@tjohnson31415 @simon-mo
[Bugfix] handle hf_config with architectures == None (vllm-project#3982)
3c81529 
Signed-off-by: Travis Johnson <[email protected]>
Co-authored-by: Simon Mo <[email protected]>
@youkaichao
[Core][Refactor] move parallel_utils into vllm/distributed (vllm-proj…
f2c4427 
…ect#3950)

[WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators (vllm-project#3950)
@rkooo567
[Core][5/N] Fully working chunked prefill e2e (vllm-project#3884)
ffb2afc
@youkaichao
[Core][Model] torch.compile for layernorm in commandr (vllm-project#3985
35295f1 
)

[Core][Model] Use torch.compile to accelerate layernorm in commandr (vllm-project#3985)
@rkooo567 @simon-mo
[Test] Add xformer and flash attn tests (vllm-project#3961)
93ee4e5 
Co-authored-by: Simon Mo <[email protected]>
@jikunshang
[Misc] Add indirection layer for custom ops (vllm-project#3913)
8b19d86
@youkaichao
[Doc][Installation] delete python setup.py develop (vllm-project#3989)
9d509bb
@ywang96
[Kernel] Fused MoE Config for Mixtral 8x22 (vllm-project#4002)
1644532
@kingljl
punica fix-bgmv-kernel-640 (vllm-project#4007)
f05fb52
@bigPYJ1151
[Hardware][Intel] Isolate CPUModelRunner and ModelRunner for better m…
33a59a3 
…aintenance (vllm-project#3824)
@Yard1
[Core] Set linear_weights directly on the layer (vllm-project#3977)
1d859cc
@youkaichao
[Core] init_distributed_environment align with init_process_group(vll…
9f93716 
…m-project#4014)

[Core][Distributed] make init_distributed_environment compatible with init_process_group (vllm-project#4014)
@robertgshaw2-redhat
Copy link
Collaborator

There was a change to the API for create_weights and apply_weights. Updated SparseW16A16LinearMethod to conform to this API.

Passing failed tests locally.

@SageMoore SageMoore merged commit c46ca5e into main Apr 15, 2024
6 of 9 checks passed
@SageMoore SageMoore deleted the upstream-sync-2024-04-12 branch April 15, 2024 13:26
@robertgshaw2-redhat robertgshaw2-redhat restored the upstream-sync-2024-04-12 branch April 15, 2024 13:34
@robertgshaw2-redhat
Copy link
Collaborator

just going to keep these branches around

@robertgshaw2-redhat robertgshaw2-redhat mentioned this pull request Apr 22, 2024
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@SageMoore SageMoore SageMoore approved these changes

@mgoin mgoin Awaiting requested review from mgoin

@rgreenberg1 rgreenberg1 Awaiting requested review from rgreenberg1

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.