Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for MPS on Apple silicon #233

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

rampadc
Copy link

@rampadc rampadc commented Mar 10, 2025

Tested on Macbook Pro M1 Max.

  • Added new virtual environment setup for MPS (start-mps_mac.sh)
  • Updated device detection logic to properly handle MPS devices
  • Enabled MPS fallback for unsupported operations

Notes

  • Only the aten::angle operation falls back to CPU, all other operations utilize MPS
  • Environment variables set automatically in startup script:
    • DEVICE_TYPE=mps
    • PYTORCH_ENABLE_MPS_FALLBACK=1

@fireblade2534
Copy link
Collaborator

would it be posible to also add support in the form of a docker container?

@rampadc
Copy link
Author

rampadc commented Mar 10, 2025

@fireblade2534 I'm not sure, maybe, but that will require a separate PR.

Docker on Mac doesn't support GPU, but Podman does, through Vulkan GPU APIs.

The only Pytorch reference for Vulkan I can find is https://pytorch.org/executorch/stable/native-delegates-executorch-vulkan-delegate.html

Or maybe Vulkan on onnx: microsoft/onnxruntime#21917

@fireblade2534
Copy link
Collaborator

ah ok

Copy link
Collaborator

@fireblade2534 fireblade2534 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a mac so I can't test this but it looks good

Copy link
Owner

@remsky remsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, will run a few checks when I'm at my Mac too, and see if I can bring a docker container around

export PYTORCH_ENABLE_MPS_FALLBACK=1

# Run FastAPI with GPU extras using uv run
uv pip install -e .
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Script looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants