Xformers wheel for PyTorch 2.0+cu117 on Windows

[aliencaocao]

Built with:
Windows 10
Python 3.9.13 (Python 3.10.9 also available below)
CUDA 11.8
torch 2.0.0.dev20221223+cu117 (latest Torch 2.0 dev on 23 Dec)

I have built xformers latest master (facebookresearch/xformers@e163309) on PyTorch 2.0. For anyone who would like to try out and see if there is performance improvements (esp with torch 2.0), you can download it here:

Python 3.9: https://1drv.ms/u/s!AvJPuRJUdWx_8hbpWdFpr234H5e_?e=eScCID
Python 3.10: https://1drv.ms/u/s!AvJPuRJUdWx_8hi5p0ecOiV3yJtT?e=Nd6rE5

You will need to install this version of pytorch:

Python 3.9 wheel: pip3 install numpy --pre torch==2.0.0.dev20221223+cu117 torchvision torchaudio --force-reinstall --extra-index-url https://download.pytorch.org/whl/nightly/cu117
Python 3.10 wheel: pip3 install numpy --pre torch==2.0.0.dev20221224+cu117 torchvision torchaudio --force-reinstall --extra-index-url https://download.pytorch.org/whl/nightly/cu117

I am running a RTX 3080Ti, and I am seeing notable speed improvements just by drop-in replacing torch 2.0 and the new xformers built with it, from 7.5it/s to 8.1it/s (768x768 Euler)

NOTE: Although the version of my wheel display as 0.0.15+e163309.d20221224, it is actually newer than all the current 0.0.16dev wheels. The 0.0.16 ones are just folks at xformers modifying the version ahead of build temporarily, while I did not do this.

[Nyaster]

Hi, i have problem with using your command, can you help me solving this problem?

[aliencaocao]

you need to replace the line in launch.py

[Cubey42]

before I destroy everything, do you mean line 161 and change pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113 to the one copied above?

[aliencaocao]

Yes

[Cubey42]

great, now I download the .whl you provided does it need to go anywhere or do I simply do the cmd install you mentioned? pip install xformers-0.0.15+e163309.d20221224-cp39-cp39-win_amd64.whl?

[aliencaocao]

It need to be in the same directory as your cmd. So if you see C:/users/something, then the file has to be in this folder too

[Revasekard]

Ok so I've edited my launch.py to " pip3 install numpy --pre torch==2.0.0.dev20221223+cu117 torchvision torchaudio --force-reinstall --extra-index-url https://download.pytorch.org/whl/nightly/cu117" which seems to install correctly.

I've extracted the files from the xformers whl and dumped them into "venv\Lib\site-packages" which isn't working out.

[aliencaocao]

You need to download the whl file i provided. Then open cmd at the same folder as your whl file, and pip install xformers-0.0.15+e163309.d20221224-cp39-cp39-win_amd64.whl. Do not follow the given name, it is different

[Revasekard]

Nada, just tells me the wheel is not supported on my current platform. Running win11.

Will probably wait till things are more streamlined for someone like me.

Thanks for the support boss.

[aliencaocao]

You have to be running on windows python 3.9

[Revasekard]

Alrighty will try that later, thanks!

[Revasekard]

Awesome I got everything to work. I was worried when everything seemed the same after installation. But dug in the wiki abit to see I had to change the VENV_DIR to "set VENV_DIR=-" .

Will have to work abit to see the changes in performance, doing a few quick generates feels like it maybe 10% ish?

Appreciate the support and patience.

[ice051128]

Built with: Windows 10 Python 3.9.13 CUDA 11.8 torch 2.0.0.dev20221223+cu117 (latest Torch 2.0 dev on 23 Dec)

I have built xformers latest master (facebookresearch/xformers@e163309) on PyTorch 2.0. For anyone who would like to try out and see if there is performance improvements (esp with torch 2.0), you can download it here: https://1drv.ms/u/s!AvJPuRJUdWx_8hbpWdFpr234H5e_?e=eScCID You will need to install this version of pytorch using pip3 install numpy --pre torch==2.0.0.dev20221223+cu117 torchvision torchaudio --force-reinstall --extra-index-url https://download.pytorch.org/whl/nightly/cu117

I am running a RTX 3080Ti, and I am seeing notable speed improvements just by drop-in replacing torch 2.0 and the new xformers built with it, from 7.5it/s to 8.1it/s.

NOTE: Although the version of my wheel display as 0.0.15+e163309.d20221224, it is actually newer than all the current 0.0.16dev wheels. The 0.0.16 ones are just folks at xformers modifying the version ahead of build temporarily, while I did not do this.

Is this safe to use with python 3.10.6?

[aliencaocao]

No, i do not have it installed, and have not enough space to install, since it will conflict with with my existing python. If you want, you can just build yourself, it is very simple

[Cubey42]

are you interested in walking us through it? I'm also on 3.10.6

[aliencaocao]

The wiki is detailed enough, i do not have python3.10 installed myself so i cannot walk through.

[Cubey42]

so which repo should I be cloning to run command? is it the xformers repo?

[aliencaocao]

Yes and that info is in the wiki

[ataa]

Where can I find the required steps to build the latest pytorch and xformers for Python 3.10.6 on Windows 10?

[aliencaocao]

Wiki

[re-unknown]

I'm a little off topic, but I decided to do this on Linux, so I installed torch2.0 with python 3.10.8, built a new virtual environment and started it (without the xformres argument yet), and I was wondering if this error would go away if I build and install xformers?
Or is it a compatibility issue?
I am not very familiar with python... 😭

Launching Web UI with arguments: --deepdanbooru --gradio-img2img-tool color-sketch
WARNING:root:Pytorch pre-release version 2.0.0.dev20221223+cu117 - assuming intent to test it
WARNING:root:Pytorch pre-release version 2.0.0.dev20221223+cu117 - assuming intent to test it
No module 'xformers'. Proceeding without it.
Error setting up CodeFormer:
Traceback (most recent call last):
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/modules/codeformer_model.py", line 38, in setup_model
    from facelib.utils.face_restoration_helper import FaceRestoreHelper
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 7, in <module>
    from facelib.detection import init_detection_model
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/__init__.py", line 11, in <module>
    from .yolov5face.face_detector import YoloDetector
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py", line 20, in <module>
    IS_HIGH_VERSION = tuple(map(int, torch.__version__.split('+')[0].split('.'))) >= (1, 9, 0)
ValueError: invalid literal for int() with base 10: 'dev20221223'

[aliencaocao]

I'm a little off topic, but I decided to do this on Linux, so I installed torch2.0 with python 3.10.8, built a new virtual environment and started it (without the xformres argument yet), and I was wondering if this error would go away if I build and install xformers?
Or is it a compatibility issue?
I am not very familiar with python... 😭

Launching Web UI with arguments: --deepdanbooru --gradio-img2img-tool color-sketch
WARNING:root:Pytorch pre-release version 2.0.0.dev20221223+cu117 - assuming intent to test it
WARNING:root:Pytorch pre-release version 2.0.0.dev20221223+cu117 - assuming intent to test it
No module 'xformers'. Proceeding without it.
Error setting up CodeFormer:
Traceback (most recent call last):
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/modules/codeformer_model.py", line 38, in setup_model
    from facelib.utils.face_restoration_helper import FaceRestoreHelper
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 7, in <module>
    from facelib.detection import init_detection_model
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/__init__.py", line 11, in <module>
    from .yolov5face.face_detector import YoloDetector
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py", line 20, in <module>
    IS_HIGH_VERSION = tuple(map(int, torch.__version__.split('+')[0].split('.'))) >= (1, 9, 0)
ValueError: invalid literal for int() with base 10: 'dev20221223'

This error is purely due to pytorch version being develop, you can safely ignore, it does nothing harmful. It is unrelated to xformers.

[re-unknown]

Thank you!
Now I can concentrate on updating xformers and cuda!

[re-unknown]

I successfully built and installed xformers and can now use it! (It was about 10% faster than before).
Thanks for the useful information ❤
By the way, is Cuda installed on linux referenced in the build? Is pytorch 11.7 referenced 🤔?

[aliencaocao]

Yes

[ice051128]

I'm a little off topic, but I decided to do this on Linux, so I installed torch2.0 with python 3.10.8, built a new virtual environment and started it (without the xformres argument yet), and I was wondering if this error would go away if I build and install xformers? Or is it a compatibility issue? I am not very familiar with python... 😭

Launching Web UI with arguments: --deepdanbooru --gradio-img2img-tool color-sketch
WARNING:root:Pytorch pre-release version 2.0.0.dev20221223+cu117 - assuming intent to test it
WARNING:root:Pytorch pre-release version 2.0.0.dev20221223+cu117 - assuming intent to test it
No module 'xformers'. Proceeding without it.
Error setting up CodeFormer:
Traceback (most recent call last):
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/modules/codeformer_model.py", line 38, in setup_model
    from facelib.utils.face_restoration_helper import FaceRestoreHelper
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 7, in <module>
    from facelib.detection import init_detection_model
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/__init__.py", line 11, in <module>
    from .yolov5face.face_detector import YoloDetector
  File "/home/ubuntu-wsl2/git/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py", line 20, in <module>
    IS_HIGH_VERSION = tuple(map(int, torch.__version__.split('+')[0].split('.'))) >= (1, 9, 0)
ValueError: invalid literal for int() with base 10: 'dev20221223'

I also got the same error man. Did u actually test it? Face restoration with codeformer no longer works with this

[aliencaocao]

Dear all, since many of you have requested for python 3.10 wheel, here it is:
https://1drv.ms/u/s!AvJPuRJUdWx_8hi5p0ecOiV3yJtT?e=Nd6rE5
Same build config/environment as my OP but with python 3.10 and torch 1224dev (the 3.9 one is built with torch 1223dev)

[aliencaocao]

Its probably because you have base-prefix in your venv cfg set to a python installation in D drive, which is fine, unless you dont actually have a python installed there.
Try adding python -m in front of the pip command

[ice051128]

Its probably because you have base-prefix in your venv cfg set to a python installation in D drive, which is fine, unless you dont actually have a python installed there. Try adding python -m in front of the pip command

I don't have a base-prefix in my venv.cfg and changing this line in activate.bat doesn't work

The python -m in front of pip command works, but I need to use pip instead of pip3 in the venv for some reason

[hithereai]

I used pip instead of pip3 as well, np with that. just aliases.

[ice051128]

I just updated successfully but it just gives this error or warning on launch

After

Before

There's only 2% performance difference when generating 768x1024 with live preview and hi res fix turned on. Is it possible to get rid of the trition error?? Also, it breaks face restoration with codeformer as it doesn't work anymore

[aliencaocao]

Triton error is expected on windows and does not affect anything. You can ignore it.
The dev int() base 10 thing is also expected and can be ignored
If you have live preview, it will introduce CPU bottleneck and the speed up will be wasted

[hithereai]

constant 30-33 it/s on my 4090 using torch 2.0 and your python 3.10 xformers whl.

It was 25-29 varying speed before, so it's great.
I also tested 512/768, and it's sitting consistently at 25 it/s, compared to 20-21 it/s before.

[LIQUIDMIND111]

hey man, its me liquidmind from discord deforum, can you teach me YOUR way to update this? since some times before your explanations help me a lot... maybe this time

[ice051128]

@aliencaocao I think u need to add an extra instruction to remove the dev string in the version.py for torch after the update to 2.0, otherwise it breaks face restoration from codeformer entirely. See the error below trition.

[hithereai]

This will remove the triton warning on startup if you care about it. Shouldn't matter.

[ice051128]

This will remove the triton warning on startup if you care about it. Shouldn't matter.

Thx man!

[aliencaocao]

It already exists in my draft PR #5965, as for the error, it will go away by itself once torch releases stable 2.0 wheels.
If you want to do a local fix, see 698d7f7#diff-e093b909cce8fa9f0d7a77571607f8a1f9733c18d6e68ca2d03699c962df6fb5 changes in webui.py

[re-unknown]

I just noticed that the instructions for 3.10 have been added and when I tried to build and install again, I received an error saying that the cuda versions are different and should be matched. I downgraded to 11.7. as per the instructions, but I thought it a bit strange that I did not get this error yesterday, so I am writing it down here, apparently it is more complicated than I thought 🤔.

[ice051128]

Yesterday I built pytorch cuda 11.7 with cuda 11.3, then updated cuda to 12.0. Then today I did what I mentioned earlier. Well, it's not a solved problem, or even a big problem, so you can ignore this comment, I'm really just curious 😅.

By the way, if you feel uncomfortable with my conversation, it is because I am using Deepl translation, I am a Japanese who cannot speak English... 😭

U can use it with cuda 11.8 instead

[aliencaocao]

Oh you cannot build with cuda 12.0 because 11.x and 12.0 is not compatible

Is it normal that u can use cu118 with pytorch 2.0? Just drop the dll files and they work

yes, and please get this clear: what you are referring to as 'drop the dll files' is NOT cuda but cudnn. Cuda is not replacbable as it is built with pytorch. Pytorch currently only have cu117 wheels. Cudnn you can use whichever you want as long as it is for cuda 11.x

[kakaxixx]

when I upgraded PyTorch 2.0+cu117,I found dreambooth can't train, No matter how many steps I trained, the new model is same as Original model, but if I use the old version, dreambooth is normal. Can you help me solve this problem? Thank you.

[aliencaocao]

when I upgraded PyTorch 2.0+cu117,I found dreambooth can't train, No matter how many steps I trained, the new model is same as Original model, but if I use the old version, dreambooth is normal. Can you help me solve this problem? Thank you.

I cannot help you as this is SD webui and not dreambooth. You should contact dreambooth authors

[kakaxixx]

thank you, I might know the reason, because cu117 or torch2.0 has One does not support dreambooth

[LIQUIDMIND111]

ok, is there any chance we can install this on a GIT PULL update from automatic1111?

[aliencaocao]

No, this is based on torch 2.0, it is not automatically installed by this repo yet, you need to manual install

[Echostorm44]

Awesome stuff, seeing about 8 it/s improvement on my 4090. Don't forget to update your cudnn libs to the torch folder when you finish all the other steps above.

[brucethemoose]

Everyone in this thread should check https://pytorch.org/blog/compromised-nightly-dependency/

[hithereai]

Everyone on Linux, windows users aren't affected

[bbecausereasonss]

Is there a 4 dummies on this step by step?

[aliencaocao]

Sorry because this is highly experimental and have potential to contain many bugs, it would only be suitable for users with some knowledge to test it out.

[ataa]

I installed it last week and it worked well, got around 7% better performance on 3070, Now I tried to install it again but that nightly build of torch seems to be missing from https://download.pytorch.org/whl/nightly/cu117 (related to hacking incident?)

Now I get this error :

RuntimeError: Couldn't install torch.
Command: "C:\SD\stable-diffusion-webui\venv\Scripts\python.exe" -m pip install numpy --pre torch==2.0.0.dev20221224+cu117 torchvision torchaudio --force-reinstall --extra-index-url https://download.pytorc
h.org/whl/nightly/cu117
Error code: 1
stdout: Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/nightly/cu117
Collecting numpy
  Downloading numpy-1.24.1-cp310-cp310-win_amd64.whl (14.8 MB)
     --------------------------------------- 14.8/14.8 MB 93.0 MB/s eta 0:00:00

stderr: ERROR: Could not find a version that satisfies the requirement torch==2.0.0.dev20221224+cu117 (from versions: 1.11.0, 1.12.0, 1.12.1, 1.13.0.dev20220719+cu117, 1.13.0.dev20220720+cu117, 1.13.0.dev
20220722+cu117, 1.13.0.dev20220723+cu117, 1.13.0.dev20220724+cu117, 1.13.0.dev20220725+cu117, 1.13.0.dev20220726+cu117, 1.13.0.dev20220727+cu117, 1.13.0.dev20220728+cu117, 1.13.0.dev20220729+cu117, 1.13.0
.dev20220730+cu117, 1.13.0.dev20220731+cu117, 1.13.0.dev20220801+cu117, 1.13.0.dev20220802+cu117, 1.13.0.dev20220803+cu117, 1.13.0.dev20220804+cu117, 1.13.0.dev20220805+cu117, 1.13.0.dev20220806+cu117, 1.
13.0.dev20220807+cu117, 1.13.0.dev20220808+cu117, 1.13.0.dev20220809+cu117, 1.13.0.dev20220810+cu117, 1.13.0.dev20220811+cu117, 1.13.0.dev20220812+cu117, 1.13.0.dev20220813+cu117, 1.13.0.dev20220814+cu117
, 1.13.0.dev20220815+cu117, 1.13.0.dev20220816+cu117, 1.13.0, 1.13.1, 2.0.0.dev20221231+cu117, 2.0.0.dev20230101+cu117, 2.0.0.dev20230102+cu117, 2.0.0.dev20230103+cu117, 2.0.0.dev20230104+cu117, 2.0.0.dev
20230105+cu117, 2.0.0.dev20230106+cu117)
ERROR: No matching distribution found for torch==2.0.0.dev20221224+cu117

[aliencaocao]

yea i dont think pytorch keep the nightly wheels for so long..

[TheForgotten69]

You can download the whl on their official website: https://download.pytorch.org/whl/nightly/torch/

[hithereai]

Here's the latest whl, compiled on xformers commit #6cd1b36 from today, pytorch 2 from yesterday (01062023), python 3.10.6, cuda 1.17, windows 11 22h2, and rtx 4090:

https://easyupload.io/sbmjuo

Tested working on an auto1111 env with the same latest pytorch nighly. Getting 33-35 it/s for a single 512*512 euler_a img on a 4090 as expected.

And a bonus: virustotal scans for the paranoids:
My latest version from this comment: https://www.virustotal.com/gui/file/80b6f15b6495e93876fb1900670d94acf3fcf4ea2038ead0042e044336abb429/detection

Original uploaded version by @aliencaocao:
https://www.virustotal.com/gui/file-analysis/ZDcyYzE4NTE2ZGFmYWZlZjEyZTNhOTMwMjQ3OGNkOWM6MTY3MzExNTY5Nw==

Both have the exact same generic PUA flag but that's nothing.

[hithereai]

Update: when I try to install pt2 now it won't install it with cuda enabled. Anyone have any insights? I also tried to pip install the version I used a few weeks back and still no luck

[aliencaocao]

It has nothing to do with torch...its xformers. You do not have to build torch whatsoever, what you need is building xformers. Each xformer can only be used with the version of torch that it is built with (the version that you install when building xformer)

[hithereai]

@aliencaocao the command pip install --pre torch --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu117 won't yield torch with cuda, just with cpu. I'm testing the same with cu118 rn.

[hithereai]

Tried to run pip install --pre torch --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu118

got this when launching webui, as cuds is not available:

[hithereai]

pip install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cu118 this one worked

[pcunwa]

pip install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cu118 this one worked

This did not work for me.
Could you tell me the version of each torch, torchvision and torchaudio as I want to download and install them manually?

[hithereai]

Xformers wheel that was compiled on pt2 with cu118 from today (26-01-23):

https://easyupload.io/hhhy2a

Compiled on win 11 22h2, python 3.10.9, cuda118 and rtx 4090.

[lifei]

file is not found

[hithereai]

Xformers v17, compiled on pt2 from 06-02-23, on win 11, python 3.10.9, cuda 11.8 and 4090:

https://easyupload.io/w2mq93

[upperm0z]

Tried the exact same build, it gives an error. win11 python 3.10.9 cu118 rtx3090

[FurkanGozukara]

24.) Automatic1111 Web UI - PC - Free
For downgrade to older version if you don't like Torch 2 : first delete venv, let it reinstall, then activate venv and run this command pip install -r "path_of_SD_Extension\requirements.txt"
How To Install New DREAMBOOTH & Torch 2 On Automatic1111 Web UI PC For Epic Performance Gains Guide

this method installs latest cuda dll files too

test py

import torch

print(f"torch {torch.__version__}, cuda {torch.version.cuda}, cudnn {torch.backends.cudnn.version()}")