Unsupported cuda toolkit version: 12.8

[hedgetenders]

Minimum reproducible example

• Install cuda toolkit 12.8
• cargo build --release --features "cuda flash-attn" (or just cuda)

Error

What is the error? If the error is very long, please try to extract an excerpt of it and post a link to a Gist

Other information

Please specify:

• Operating system: Windows
• GPU or accelerator information

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jan_15_19:38:46_Pacific_Standard_Time_2025
Cuda compilation tools, release 12.8, V12.8.61
Build cuda_12.8.r12.8/compiler.35404655_0

nvidia-smi

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 571.96                 Driver Version: 571.96         CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090      WDDM  |   00000000:01:00.0  On |                  Off |
| 31%   37C    P0             66W /  450W |    1957MiB /  24564MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

Latest commit or version

master

[EricLBuehler]

@hedgetenders this comes from upstream: coreylowman/cudarc#324