This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Upstream sync 2024 04 12#183

Merged

SageMoore merged 44 commits intomainfrom upstream-sync-2024-04-12

Apr 15, 2024

+5,193-1,306

Commits on Apr 12, 2024

[Bugfix] Added Command-R GPTQ support (vllm-project#3849 )

authored and
andy-neuma
committed
[Bugfix] Enable Proper attention_bias Usage in Llama Model Configuration (vllm-project#3767 )

authored and
andy-neuma
committed
[Hotfix][CI/Build][Kernel] CUDA 11.8 does not support layernorm optimizations (vllm-project#3782 )

mawong-amd
authored and
andy-neuma
committed
[BugFix][Model] Fix commandr RoPE max_position_embeddings (vllm-project#3919 )

esmeetu
authored and
andy-neuma
committed
[Core] separate distributed_init from worker (vllm-project#3904 )

youkaichao
authored and
andy-neuma
committed
[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" (vllm-project#3837 )

cadedaniel
authored and
andy-neuma
committed
[Bugfix] Fix KeyError on loading GPT-NeoX (vllm-project#3925 )

jsato8094
authored and
andy-neuma
committed
[ROCm][Hardware][AMD] Use Triton Kernel for default FA on ROCm (vllm-project#3643 )

authored and
andy-neuma
committed
[Misc] Avoid loading incorrect LoRA config (vllm-project#3777 )

jeejeelee
authored and
andy-neuma
committed
[Benchmark] Add cpu options to bench scripts (vllm-project#3915 )

zedong-peng
authored and
andy-neuma
committed
[Bugfix] fix utils.py/merge_dict func TypeError: 'type' object is not subscriptable (vllm-project#3955 )

authored and
andy-neuma
committed
[Bugfix] Fix logits processor when prompt_logprobs is not None (vllm-project#3899 )

huyiwen
authored and
andy-neuma
committed
[Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (vllm-project#3876 )

tjohnson31415
authored and
andy-neuma
committed
[Bugfix][ROCm] Add numba to Dockerfile.rocm (vllm-project#3962 )

WoosukKwon
authored and
andy-neuma
committed
[Model][AMD] ROCm support for 256 head dims for Gemma (vllm-project#3972 )

jamestwhedbee
authored and
andy-neuma
committed
[Doc] Add doc to state our model support policy (vllm-project#3948 )

authored and
andy-neuma
committed
[Bugfix] Remove key sorting for guided_json parameter in OpenAi compatible Server (vllm-project#3945 )

dmarasco
authored and
andy-neuma
committed
[Doc] Fix getting stared to use publicly available model (vllm-project#3963 )

fpaupier
authored and
andy-neuma
committed
[Bugfix] handle hf_config with architectures == None (vllm-project#3982 )

authored and
andy-neuma
committed
[Core][Refactor] move parallel_utils into vllm/distributed (vllm-project#3950 )

youkaichao
authored and
andy-neuma
committed
[Core][5/N] Fully working chunked prefill e2e (vllm-project#3884 )

rkooo567
authored and
andy-neuma
committed
[Core][Model] torch.compile for layernorm in commandr (vllm-project#3985 )

youkaichao
authored and
andy-neuma
committed
[Test] Add xformer and flash attn tests (vllm-project#3961 )

authored and
andy-neuma
committed
[Misc] Add indirection layer for custom ops (vllm-project#3913 )

jikunshang
authored and
andy-neuma
committed
[Doc][Installation] delete python setup.py develop (vllm-project#3989 )

youkaichao
authored and
andy-neuma
committed
[Kernel] Fused MoE Config for Mixtral 8x22 (vllm-project#4002 )

ywang96
authored and
andy-neuma
committed
punica fix-bgmv-kernel-640 (vllm-project#4007 )

kingljl
authored and
andy-neuma
committed
[Hardware][Intel] Isolate CPUModelRunner and ModelRunner for better maintenance (vllm-project#3824 )

bigPYJ1151
authored and
andy-neuma
committed
[Core] Set linear_weights directly on the layer (vllm-project#3977 )

Yard1
authored and
andy-neuma
committed
[Core] init_distributed_environment align with init_process_group(vllm-project#4014 )

youkaichao
authored and
andy-neuma
committed
Fix echo/logprob OpenAI completion bug (vllm-project#3441 )

authored and
andy-neuma
committed
Add extra punica sizes to support bigger vocabs (vllm-project#4015 )

Yard1
authored and
andy-neuma
committed
[BugFix] Fix handling of stop strings and stop token ids (vllm-project#3672 )

njhill
authored and
andy-neuma
committed
[Doc] Add typing hints / mypy types cleanup (vllm-project#3816 )

authored and
andy-neuma
committed
[Core] Support LoRA on quantized models (vllm-project#4012 )

jeejeelee
authored and
andy-neuma
committed
[Frontend][Core] Move merge_async_iterators to utils (vllm-project#4026 )

DarkLight1337
authored and
andy-neuma
committed
disable tests, waiting for upstream
andy-neuma
committed

Commits

Commits on Apr 12, 2024

Commits on Apr 14, 2024