Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: [CANN] backend adapts to llama.cpp dynamic backend loading mechanism #9862

Closed
4 tasks done
leo-pony opened this issue Oct 12, 2024 · 0 comments
Closed
4 tasks done
Assignees
Labels
Ascend NPU issues specific to Ascend NPUs enhancement New feature or request

Comments

@leo-pony
Copy link
Contributor

Prerequisites

  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Dynamically loadable backends framework has been added in PR(#9707). CANN backend needs to adapt to this mechanism.

Motivation

llama.cpp will be refactored to use only the backend registry API, as explained by slaren in PR (#9707). Currently, CUDA and CPU backends has implemented these interfaces.

Possible Implementation

CANN already implement the functions in these interfaces, so this should only require shuffling some code around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ascend NPU issues specific to Ascend NPUs enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants