Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README.md 20250205_aiter #407

Merged
merged 3 commits into from
Feb 6, 2025
Merged

Update README.md 20250205_aiter #407

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d57b480
Update README.md 20250205_aiter
arakowsk-amd Feb 6, 2025
4da6071
whitespace
arakowsk-amd Feb 6, 2025
796a793
adding VLLM_USE_AITER=0 advice
arakowsk-amd Feb 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/dev-docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,18 @@ The pre-built image includes:

- ROCm™ 6.3.1
- vLLM 0.6.6
- PyTorch 2.6dev (nightly)
- PyTorch 2.7dev (nightly)

## Pull latest Docker Image

Pull the most recent validated docker image with `docker pull rocm/vllm-dev:main`

## What is New

20250205_aiter:
- [AITER](https://github.com/ROCm/aiter) support
- Performance improvement for custom paged attention
- Reduced memory overhead bug fix
20250124:
- Fix accuracy issue with 405B FP8 Triton FA
- Fixed accuracy issue with TP8
Expand Down Expand Up @@ -475,8 +479,9 @@ To reproduce the release docker:
```bash
git clone https://github.com/ROCm/vllm.git
cd vllm
git checkout 8e87b08c2a284c1a20eb3d8e0fbdc84918bf27dc
git checkout 9dc3394c9ee4da250be28d7bd08babf098d51081
docker build -f Dockerfile.rocm -t <your_tag> --build-arg BUILD_HIPBLASLT=1 --build-arg USE_CYTHON=1 .
export VLLM_USE_AITER=0
```

### AITER
Expand Down