Merge pull request #47 from fivetran/update_deps_versions

update dependencies

CHANGELOG.md

@@ -1,12 +1,29 @@
 # dbt_google_ads_source v0.10.0
-[PR #43](https://github.com/fivetran/dbt_google_ads_source/pull/43) includes the following updates:
-## Feature update 🎉
-- Unioning capability! This adds the ability to union source data from multiple google_ads connectors. Refer to the [README](https://github.com/fivetran/dbt_google_ads_source/blob/main/README.md) for more details.
 
-## Under the hood 🚘
+## Dependency Updates
+- Removes the dependency on [dbt-expectations](https://github.com/calogica/dbt-expectations/releases). Specifically we removed the `dbt_expectations.expect_column_values_to_not_match_regex_list` test. ([PR #47](https://github.com/fivetran/dbt_google_ads_source/pull/47))
+
+## Feature Update 🎉
+- Unioning capability! This adds the ability to union source data from multiple google_ads connectors. Refer to the [README](https://github.com/fivetran/dbt_google_ads_source/blob/main/README.md) for more details. ([PR #43](https://github.com/fivetran/dbt_google_ads_source/pull/43))
+
+## Under the Hood 🚘
 - Updated tmp models to union source data using the `fivetran_utils.union_data` macro. 
-- To distinguish which source each field comes from, added `source_relation` column in each staging model and applied the `fivetran_utils.source_relation` macro.
-- Updated tests to account for the new `source_relation` column.
+- To distinguish which source each field comes from, added `source_relation` column in each staging model and applied the `fivetran_utils.source_relation` macro. ([PR #43](https://github.com/fivetran/dbt_google_ads_source/pull/43))
+- Updated tests to account for the new `source_relation` column. ([PR #43](https://github.com/fivetran/dbt_google_ads_source/pull/43))
+
+# dbt_google_ads_source v0.9.5
+## Rollback
+[PR #46](https://github.com/fivetran/dbt_google_ads_source/pull/46) rolls back [PR #45](https://github.com/fivetran/dbt_google_ads_source/pull/45) 
+
+- This was causing conflicting dbt-expectation versions because of the version required in other packages.
+
+
+# dbt_google_ads_source v0.9.4
+
+[PR #45](https://github.com/fivetran/dbt_google_ads_source/pull/45) includes the following updates:
+## Under the Hood:
+- Updates the [dbt-expectations](https://github.com/calogica/dbt-expectations/releases) dependency to the latest version.
+- Updates the [DECISIONLOG](DECISIONLOG.md) to clarify why there exist differences among aggregations across different grains.
 
 # dbt_google_ads_source v0.9.3
 
@@ -138,4 +155,4 @@ PR [#29](https://github.com/fivetran/dbt_google_ads_source/pull/29) includes the
 - [NoToWarAlways](https://github.com/NoToWarAlways) ([#19](https://github.com/fivetran/dbt_google_ads_source/pull/19))
 
 # dbt_google_ads_source v0.1.0 -> v0.4.0
-Refer to the relevant release notes on the Github repository for specific details for the previous releases. Thank you!
+Refer to the relevant release notes on the Github repository for specific details for the previous releases. Thank you!

DECISIONLOG.md

@@ -2,4 +2,10 @@
 ## Ads Associated with Multiple Ad Groups
 It was discovered within the source data that a single Ad can be associated with multiple ad groups on any given day. Because of this, it was determined that the `is_most_recent_record` logic within the `stg_google_ads__ad_history` model needed to account for the `ad_group_id` as well as the individual `ad_id`. As a result, the most recent record of an ad could possibly contain a unique combination of the `ad_id` and the `ad_group_id`.
 
-This logic was only applied to the `stg_google_ads__ad_history` model as it was discovered this relationship was unique to ads and ad groups. If you experience this relationship among any of the other ad hierarchies, please open and [issue](https://github.com/fivetran/dbt_google_ads_source/issues/new?assignees=&labels=bug%2Ctriage&template=bug-report.yml&title=%5BBug%5D+%3Ctitle%3E) and we can continue the discussion!
+This logic was only applied to the `stg_google_ads__ad_history` model as it was discovered this relationship was unique to ads and ad groups. If you experience this relationship among any of the other ad hierarchies, please open and [issue](https://github.com/fivetran/dbt_google_ads_source/issues/new?assignees=&labels=bug%2Ctriage&template=bug-report.yml&title=%5BBug%5D+%3Ctitle%3E) and we can continue the discussion!
+
+
+## Why don't metrics add up across different grains (Ex. ad level vs campaign level)?
+Not all ads are served at the ad level. In other words, there are some ads that are served only at the ad group, campaign, etc. levels. The implications are that since not ads are included in the ad-level report, their associated spend, for example, won't be included at that grain. Therefore your spend totals may differ across the ad grain and another grain. 
+
+This is a reason why we have broken out the ad reporting packages into separate hierarchical end models (Ad, Ad Group, Campaign, and more). Because if we only used ad-level reports, we could be missing data.

README.md

@@ -26,14 +26,11 @@ To use this dbt package, you must have the following:
 - A **BigQuery**, **Snowflake**, **Redshift**, **PostgreSQL**, or **Databricks** destination.
 
 ### Databricks Dispatch Configuration
-If you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your `dbt_project.yml`. This is required in order for the package to accurately search for macros within the `dbt-labs/spark_utils` then the `dbt-labs/dbt_utils` as well as the `calogica/dbt_expectations` then the `google_ads_source` packages respectively.
+If you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your `dbt_project.yml`. This is required in order for the package to accurately search for macros within the `dbt-labs/spark_utils` then the `dbt-labs/dbt_utils`.
 ```yml
 dispatch:
   - macro_namespace: dbt_utils
     search_order: ['spark_utils', 'dbt_utils']
-
-  - macro_namespace: dbt_expectations
-    search_order: ['google_ads_source', 'dbt_expectations']
 ```
 
 ## Step 2: Install the package (skip if also using the `google_ads` transformation package)
@@ -128,12 +125,6 @@ packages:
     - package: dbt-labs/dbt_utils
       version: [">=1.0.0", "<2.0.0"]
 
-    - package: calogica/dbt_expectations
-      version: [">=0.8.0", "<0.9.0"]
-
-    - package: calogica/dbt_date
-      version: [">=0.7.0", "<0.8.0"]
-
     - package: dbt-labs/spark_utils
       version: [">=0.3.0", "<0.4.0"]
 ```

integration_tests/ci/sample.profiles.yml

@@ -16,13 +16,13 @@ integration_tests:
       pass: "{{ env_var('CI_REDSHIFT_DBT_PASS') }}"
       dbname: "{{ env_var('CI_REDSHIFT_DBT_DBNAME') }}"
       port: 5439
-      schema: google_ads_source_integration_tests_2
+      schema: google_ads_source_integration_tests_3
       threads: 8
     bigquery:
       type: bigquery
       method: service-account-json
       project: 'dbt-package-testing'
-      schema: google_ads_source_integration_tests_2
+      schema: google_ads_source_integration_tests_3
       threads: 8
       keyfile_json: "{{ env_var('GCLOUD_SERVICE_KEY') | as_native }}"
     snowflake:
@@ -33,7 +33,7 @@ integration_tests:
       role: "{{ env_var('CI_SNOWFLAKE_DBT_ROLE') }}"
       database: "{{ env_var('CI_SNOWFLAKE_DBT_DATABASE') }}"
       warehouse: "{{ env_var('CI_SNOWFLAKE_DBT_WAREHOUSE') }}"
-      schema: google_ads_source_integration_tests_2
+      schema: google_ads_source_integration_tests_3
       threads: 8
     postgres:
       type: postgres
@@ -42,13 +42,13 @@ integration_tests:
       pass: "{{ env_var('CI_POSTGRES_DBT_PASS') }}"
       dbname: "{{ env_var('CI_POSTGRES_DBT_DBNAME') }}"
       port: 5432
-      schema: google_ads_source_integration_tests_2
+      schema: google_ads_source_integration_tests_3
       threads: 8
     databricks:
       catalog: "{{ env_var('CI_DATABRICKS_DBT_CATALOG') }}"
       host: "{{ env_var('CI_DATABRICKS_DBT_HOST') }}"
       http_path: "{{ env_var('CI_DATABRICKS_DBT_HTTP_PATH') }}"
-      schema: google_ads_source_integration_tests_2
+      schema: google_ads_source_integration_tests_3
       threads: 8
       token: "{{ env_var('CI_DATABRICKS_DBT_TOKEN') }}"
       type: databricks

integration_tests/dbt_project.yml

@@ -4,7 +4,7 @@ profile: 'integration_tests'
 config-version: 2
 
 vars:
-  google_ads_schema: google_ads_source_integration_tests_2
+  google_ads_schema: google_ads_source_integration_tests_3
   google_ads_ad_stats_identifier: "ad_stats_data"
   google_ads_ad_history_identifier: "ad_history_data"
   google_ads_ad_group_history_identifier: "ad_group_history_data"
@@ -32,6 +32,3 @@ seeds:
 dispatch:
   - macro_namespace: dbt_utils
     search_order: ['spark_utils', 'dbt_utils']
-
-  - macro_namespace: dbt_expectations
-    search_order: ['google_ads_source', 'dbt_expectations']

models/stg_google_ads.yml

@@ -89,11 +89,6 @@ models:
         description: "{{ doc('is_most_recent_record') }}"
       - name: source_final_urls
         description: The original list of final urls expressed as an array. Please be aware the test used on this field is intended to warn you if you have fields with multiple urls. If you do, the `final_url` field will filter down the urls within the array to just the first. Therefore, this package will only leverage one of possibly many urls within this field array.
-        tests:
-          - dbt_expectations.expect_column_values_to_not_match_regex_list:
-              regex_list: ","
-              match_on: any
-              severity: warn
       - name: final_url
         description: The first url in the list of the urls within the `final_urls` source field.
       - name: base_url

packages.yml

@@ -1,6 +1,4 @@
 packages:
-- package: calogica/dbt_expectations
-  version: [">=0.8.0", "<0.9.0"]
 
 # - package: fivetran/fivetran_utils
 #   version: [">=0.4.0", "<0.5.0"]