Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CPU] Optimize small batch case for PagedAttention #27847

Merged
merged 4 commits into from
Dec 6, 2024
Merged

[CPU] Optimize small batch case for PagedAttention #27847

merged 4 commits into from
Dec 6, 2024

Conversation

luo-cheng2021
Copy link
Contributor

@luo-cheng2021 luo-cheng2021 commented Dec 2, 2024 •
edited
Loading

Details:

  • Generate more work items to avoid thread imbalance
  • ...

Tickets:

@github-actions github-actions bot added the category: CPU OpenVINO CPU plugin label Dec 2, 2024
@luo-cheng2021 luo-cheng2021 mentioned this pull request Dec 2, 2024
Closed
@luo-cheng2021 luo-cheng2021 marked this pull request as ready for review December 4, 2024 01:49
@luo-cheng2021 luo-cheng2021 requested review from a team as code owners December 4, 2024 01:49
@luo-cheng2021 luo-cheng2021 force-pushed the luocheng/pa_vs_sdpa_formal branch from fb28291 to 81320f5 Compare December 4, 2024 06:44
@dmitry-gorokhov
Copy link
Contributor

@zhangYiIntel Could you please take a look?

@dmitry-gorokhov dmitry-gorokhov added this to the 2025.0 milestone Dec 4, 2024
zhangYiIntel
zhangYiIntel approved these changes Dec 5, 2024
View reviewed changes
Copy link
Contributor

@zhangYiIntel zhangYiIntel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General LGTM. Minor changes to reduce redundant code.

@dmitry-gorokhov dmitry-gorokhov added this pull request to the merge queue Dec 6, 2024
Merged via the queue into openvinotoolkit:master with commit d62effb Dec 6, 2024
169 checks passed
@ilya-lavrenov ilya-lavrenov deleted the luocheng/pa_vs_sdpa_formal branch December 10, 2024 07:46
@luo-cheng2021 luo-cheng2021 mentioned this pull request Dec 31, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmitry-gorokhov dmitry-gorokhov dmitry-gorokhov approved these changes

@zhangYiIntel zhangYiIntel zhangYiIntel approved these changes

Assignees

@zhangYiIntel zhangYiIntel

Labels
category: CPU OpenVINO CPU plugin
Projects
None yet
Milestone
2025.0
Development

Successfully merging this pull request may close these issues.
3 participants
@luo-cheng2021 @dmitry-gorokhov @zhangYiIntel