Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updates and rm run results in docs #39

Merged
merged 20 commits into from
Nov 5, 2024
Merged

Conversation

fivetran-reneeli
Copy link
Contributor

@fivetran-reneeli fivetran-reneeli commented Oct 18, 2024

PR Overview

This PR will address the following Issue/Feature:
#36
#37

This PR will result in the following new package version:

v0.10.0
breaking change since this could affect runs and will result in downstream changes as well

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Breaking Change (--full-refresh required after upgrading)

  • Updates the materialization in stg_iterable__user_history from table to view in order to improve performance. A --full-refresh is required after upgrading to ensure there are no issues.

Bug Fix

  • Removes the source freshness tests in the src_iterable.yml. This was originally causing run errors for users where tables weren't present in their schema but listed in the src_iterable.yml.
  • Removes created_at from the uniqueness test for stg_iterable__event. Uniqueness is now tested solely on unique_event_id, a surrogate key made up of event_id (_fivetran_id in the raw table, which is a Fivetran-created unique identifier derived from hashing campaign_id, created_at, and event_name) and _fivetran_user_id (a Fivetran-created column derived from a hash of user_id and/or email).

Documentation

  • Updates the descriptions of timestamp-based fields. Previously they were described as milliseconds since epoch time, but they should be standard timestamps.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • [na] Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

  • This is just a removal of the source freshness tests in the src yml, allowing the models to run successfully in users environments. I also ran the downstream models with this branch, with no issue.

If you had to summarize this PR in an emoji, which would it be?

💃

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-reneeli this looks good to me, but I have one documentation request to add before approval. Let me know if you have questions. Thanks!

@fivetran-reneeli fivetran-reneeli linked an issue Oct 28, 2024 that may be closed by this pull request
4 tasks
@fivetran-reneeli fivetran-reneeli linked an issue Oct 28, 2024 that may be closed by this pull request
1 task
Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment to update the CHANGELOG. Otherwise looks good!

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM just a few final requests to update the CHANGELOG

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks good, just a few final change requests before opening for release review.

CHANGELOG.md Outdated
Comment on lines 14 to 15
## Contributors
- [@justin-fundrise](https://github.com/justin-fundrise)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is no PR linked in the source for the contribution then we shouldn't include this section in the CHANGELOG. We should only include a contributing section when there is a PR linked within the package update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok! That makes a lot of sense

fivetran-reneeli and others added 2 commits November 1, 2024 15:55
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM just one CHANGELOG edit

Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Copy link
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-reneeli Great work! A few comments, but should be good to merge after that.

fivetran-reneeli and others added 3 commits November 4, 2024 17:01
Co-authored-by: Avinash Kunnath <[email protected]>
Co-authored-by: Avinash Kunnath <[email protected]>
Co-authored-by: Avinash Kunnath <[email protected]>
@fivetran-reneeli fivetran-reneeli merged commit 269c852 into main Nov 5, 2024
7 checks passed
@fivetran-reneeli fivetran-reneeli deleted the rm_src_freshness branch November 5, 2024 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants