Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Destination tests for batch exports #28687

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

tomasfarias
Copy link
Contributor

Problem

The most common type error when interacting with batch exports are user configuration errors. This PR introduces a new framework for "Destination tests": Tests that can be called from our API to evaluate a given batch export configuration.

Changes

  • [x]: Bigquery destination tests
  • [ ]: Snowflake destination tests
  • [ ]: S3 destination tests
  • [ ]: PostgreSQL destination tests
  • [ ]: Redshift destination tests
  • [ ]: Frontend support
  • [x]: API Support

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Does this work well for both Cloud and self-hosted?

How did you test this code?

New unit tests.

@tomasfarias tomasfarias force-pushed the feat/destination-tests-for-batch-exports branch from 43f589a to 2dab705 Compare February 14, 2025 10:04
Comment on lines 529 to 530
@action(methods=["POST"], detail=True)
def test(self, request: request.Request, *args, **kwargs) -> response.Response:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this endpoint just for fetching a test (and it's list of steps) for a given batch export?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, thought about having everything in a single endpoint, but it looked too complicated. I'd be happy to rename this to something more verbose though...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it does make sense to get a list of possible tests upfront, especially if we want to add them to destinations one at a time, since the frontend could probably not even show a test button if there are no tests to run for example. Was also wondering if it could just be a GET method if it's not actually running anything

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants