Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

colexec: implement vectorized table statistics collection #54803

Closed
yuzefovich opened this issue Sep 25, 2020 · 4 comments
Closed

colexec: implement vectorized table statistics collection #54803

yuzefovich opened this issue Sep 25, 2020 · 4 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@yuzefovich
Copy link
Member

Statistics from the tables are currently collected using a combination of row-execution sampler and sampleAggregator processors which have been introduced before we began implementing the vectorized engine. I believe that stats collection will benefit noticeably from the vectorized approach (similar benefits as in the other use cases - faster execution, better memory management). In particular, I'm hoping that it will alleviate issues like #54670.

@yuzefovich yuzefovich added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Sep 25, 2020
@asubiotto
Copy link
Contributor

@RaduBerinde I think the optimizer team needs to prioritize this for 21.1 given that #54670 is an easily reproducible OOM and we've seen a lot of OOMs related to wide rows lately. We're happy to offer guidance here.

@RaduBerinde
Copy link
Member

@rytaft I thought we had hit OOMs with large rows before and had a workaround in place, do you remember the details? I think it was around dynamically adjusting the batch size or something along those lines

@rytaft
Copy link
Collaborator

rytaft commented Sep 30, 2020

Maybe you're thinking of #40850? I didn't end up merging that PR since there were some performance issues identified. Seems like we should just switch to vectorized to fix this.

@RaduBerinde
Copy link
Member

I see. Let's use the existing #41203 to track this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Development

No branches or pull requests

4 participants