Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: introduce buffer pool(aka memory manager) to avoid OOM for some opeartors #7271

Closed
xudong963 opened this issue Aug 24, 2022 · 2 comments
Assignees
Labels
C-feature Category: feature

Comments

@xudong963
Copy link
Member

Summary

When input data is large, some operators(such as join, sort and so on) will OOM.

It's necessary to introduce a memory manager to avoid OOM

  • let users specify memory size to avoid OOM
  • Memory manager in charge of spilling for join, sort, group by...
  • ...

cc @zhang2014

@FANNG1
Copy link

FANNG1 commented Aug 25, 2022

does "some operator" include table scan operator?

@zhang2014
Copy link
Member

does "some operator" include table scan operator?

We will use global allocator to track and limit memory usage. the processor level memory trace is still in the todo list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature Category: feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants