Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prune old versions also for views #522

Merged
merged 1 commit into from
Nov 27, 2023

Conversation

juliansteger-sc
Copy link
Contributor

Description

In our dbt runs we received the following error:

Number of TABLE_VERSION resources exceeds the account limit 1000000

Debugging led to the conclusion that views are creating new versions but old views are not cleaned up.

Models used to test - Optional

  • internal views that had lots of versions, after dbt run with this change they are down to last 4 glue versions

Checklist

  • You followed contributing section
  • You kept your Pull Request small and focused on a single feature or bug fix.
  • You added unit testing when necessary
  • You added functional testing when necessary

@nicor88
Copy link
Contributor

nicor88 commented Nov 27, 2023

Nice catch @juliansteger-sc , I believe that we can do something similar for incremental tables..

@juliansteger-sc
Copy link
Contributor Author

@nicor88 agreed, I didn't check this. should we do it in a follow up PR or right here?

@juliansteger-sc
Copy link
Contributor Author

@nicor88 fucntional-tests report permissioning error

@nicor88
Copy link
Contributor

nicor88 commented Nov 27, 2023

@juliansteger-sc another PR for incremental models seems about right for me. Let's keep it narrow for now.

@nicor88
Copy link
Contributor

nicor88 commented Nov 27, 2023

@juliansteger-sc the CI is being fix. Fill free to open an issue about using this solution on incremental models.

@nicor88 nicor88 merged commit 168d806 into dbt-labs:main Nov 27, 2023
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants