Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISSUE #14765 - Implement Athena Injected Partition Check #15318

Merged
merged 20 commits into from
Feb 28, 2024

Conversation

TeddyCr
Copy link
Contributor

@TeddyCr TeddyCr commented Feb 22, 2024

Describe your changes:

Depends on refactor work from #15312

Fixes #14765

  • Implement logic to check if an Athena injected partition has a configuration set in the tableProfilerConfig

Why just implement a check and throw an error message?
Athena injected partition type are use to require user to set the field in the WHERE clause of the query. This can only be applied to string column types -- hence use need to set a COLUMN-VALUE partition type in the profiler config.

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

Improvement

  • I have added tests around the new logic.
  • For connector/ingestion changes: I updated the documentation.

@TeddyCr TeddyCr enabled auto-merge (squash) February 28, 2024 09:02
@TeddyCr TeddyCr requested a review from pmbrull February 28, 2024 11:42
pmbrull
pmbrull previously approved these changes Feb 28, 2024
Copy link
Contributor

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

Copy link
Contributor

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

pmbrull
pmbrull previously approved these changes Feb 28, 2024
Copy link

@TeddyCr TeddyCr merged commit 3e83bda into open-metadata:main Feb 28, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Ingestion safe to test Add this label to run secure Github workflows on PRs UI UI specific issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Profiler - Athena metric computation error with injected projected partitions
2 participants