Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package hub prep #2

Merged
merged 12 commits into from
Feb 10, 2023
82 changes: 44 additions & 38 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,16 @@ jobs:
steps:
- checkout

- run:
run: setup_creds
command: |
echo $BIGQUERY_SERVICE_ACCOUNT_JSON > ${HOME}/bigquery-service-key.json
# fleetio note:
# the section below was written when dbt labs owned the package and supported bigquery and redshift
# we don't have test environments for redshift and bigquery, but we wanted to keep this in case we can support it in the future
# we're working on setting up a snowflake service account to use for the snowflake test, but for now we'll only run the CI job for postgres

# - run:
# run: setup_creds
# command: |
# echo $BIGQUERY_SERVICE_ACCOUNT_JSON > ${HOME}/bigquery-service-key.json


- restore_cache:
key: deps1-{{ .Branch }}
Expand All @@ -25,7 +31,7 @@ jobs:
. dbt_venv/bin/activate

pip install --upgrade pip setuptools
pip install --pre dbt-core dbt-postgres dbt-redshift dbt-snowflake dbt-bigquery
pip install --pre dbt-core dbt-postgres dbt-snowflake

mkdir -p ~/.dbt
cp integration_tests/ci/sample.profiles.yml ~/.dbt/profiles.yml
Expand All @@ -46,43 +52,43 @@ jobs:
dbt --warn-error seed --target postgres
dbt --warn-error run --target postgres --full-refresh
dbt --warn-error run --target postgres

# - run:
# name: "Run Tests - Redshift"
# command: |
# . dbt_venv/bin/activate
# echo `pwd`
# cd integration_tests
# dbt --warn-error deps
# dbt --warn-error seed --target redshift
# dbt --warn-error run --target redshift --full-refresh
# dbt --warn-error run --target redshift

# - run:
# name: "Run Tests - BigQuery"
# environment:
# BIGQUERY_SERVICE_KEY_PATH: "/home/circleci/bigquery-service-key.json"

- run:
name: "Run Tests - Redshift"
command: |
. dbt_venv/bin/activate
echo `pwd`
cd integration_tests
dbt --warn-error deps
dbt --warn-error seed --target redshift
dbt --warn-error run --target redshift --full-refresh
dbt --warn-error run --target redshift
# command: |
# . dbt_venv/bin/activate
# echo `pwd`
# cd integration_tests
# dbt --warn-error deps
# dbt --warn-error seed --target bigquery
# dbt --warn-error run --target bigquery --full-refresh
# dbt --warn-error run --target bigquery

- run:
name: "Run Tests - Snowflake"
command: |
. dbt_venv/bin/activate
echo `pwd`
cd integration_tests
dbt --warn-error deps
dbt --warn-error seed --target snowflake
dbt --warn-error run --target snowflake --full-refresh
dbt --warn-error run --target snowflake

- run:
name: "Run Tests - BigQuery"
environment:
BIGQUERY_SERVICE_KEY_PATH: "/home/circleci/bigquery-service-key.json"

command: |
. dbt_venv/bin/activate
echo `pwd`
cd integration_tests
dbt --warn-error deps
dbt --warn-error seed --target bigquery
dbt --warn-error run --target bigquery --full-refresh
dbt --warn-error run --target bigquery
# - run:
# name: "Run Tests - Snowflake"
# command: |
# . dbt_venv/bin/activate
# echo `pwd`
# cd integration_tests
# dbt --warn-error deps
# dbt --warn-error seed --target snowflake
# dbt --warn-error run --target snowflake --full-refresh
# dbt --warn-error run --target snowflake

- save_cache:
key: deps1-{{ .Branch }}
Expand Down
45 changes: 45 additions & 0 deletions .github/release-drafter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name-template: 'v$RESOLVED_VERSION'
tag-template: 'v$RESOLVED_VERSION'
template: |
# What's Changed

$CHANGES

**Full Changelog**: https://github.com/$OWNER/$REPOSITORY/compare/$PREVIOUS_TAG...v$RESOLVED_VERSION

categories:
- title: '🚀 New Features'
labels:
- 'feature'
- 'enhancement'
- title: '🐛 Bug Fixes'
labels:
- 'fix'
- 'bugfix'
- 'bug'
- title: '🧰 Maintenance'
labels:
- 'chore'
- 'docs'
- 'security'
- 'dependencies'

version-resolver:
major:
labels:
- 'major'
minor:
labels:
- 'feature'
- 'enhancement'
patch:
labels:
- 'fix'
- 'bugfix'
- 'bug'
- 'dependencies'
- 'security'
- 'docs'
- 'chore'
default: patch

29 changes: 29 additions & 0 deletions .github/workflows/release-drafter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Release Drafter

on:
push:
branches:
- main

permissions:
contents: read

jobs:
update_release_draft:
permissions:
# write permission is required to create a github release
contents: write
# write permission is required for autolabeler
# otherwise, read permission is required at least
pull-requests: write
runs-on: ubuntu-latest
steps:

# Drafts your next Release notes as Pull Requests are merged into "main"
- uses: release-drafter/release-drafter@v5
# (Optional) specify config name to use, relative to .github/. Default: release-drafter.yml
# with:
# config-name: my-config.yml
# disable-autolabeler: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
> ⛔🏚️ This package is no longer developed or maintained by dbt Labs. If you're interested in forking/adopting it, reach out in #package-ecosystem on dbt Slack.
> ⛔🏚️ This package is not officially related to Segment and is maintained by [Fleetio](https://fleetio.com). CI jobs run on PRs will only test for postgres compatibility. We're working on setting up a service account to support tests for Snowflake. Please submit an issue for any bugs/feature requests related to Redshift or BigQuery and we'll figure out how to help!

# dbt-segment
This [dbt package](https://docs.getdbt.com/docs/package-management):
Expand Down Expand Up @@ -54,4 +54,4 @@ the package to run successfully.
This package has been tested on Redshift, Snowflake, BigQuery, and Postgres.

### Contributing
Additional contributions to this repo are very welcome! Check out [this post](https://discourse.getdbt.com/t/contributing-to-a-dbt-package/657) on the best workflow for contributing to a package. All PRs should only include functionality that is contained within all Segment deployments; no implementation-specific details should be included.
Additional contributions to this repo are very welcome! Check out [this post](https://discourse.getdbt.com/t/contributing-to-a-dbt-package/657) on the best workflow for contributing to a package. All PRs should only include functionality that is contained within all Segment deployments; no implementation-specific details should be included. CI jobs run on PRs will only test for postgres compatibility.
4 changes: 2 additions & 2 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ seed-paths: ["seeds"]
profile: 'integration_tests'

vars:
segment:
dbt_segment:
segment_page_views_table: "{{ ref('example_segment_pages') }}"

seeds:
+quote_columns: false
+quote_columns: "{{ true if target.type in ('redshift','postgres') else false }}"
4 changes: 2 additions & 2 deletions integration_tests/seeds/example_segment_pages.csv
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
id,anonymous_id,user_id,received_at,sent_at,timestamp,url,path,title,search,referrer,context_campaign_source,context_campaign_medium,context_campaign_name,context_campaign_term,context_campaign_content,context_ip,context_user_agent
ajs-9527e97fc714abe02a23b3d24cf09a36,507f191e810c19729de860ea,97980cfea0067,2015-02-23 22:28:55,2015-02-23 22:28:55,2015-02-23 22:28:55,https://www.getdbt.com/,/product,Product,?name=ferret,https://www.google.com/,facebook,social,autumn_collection_2015,foo,bar,8.8.8.8,"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36"
id,anonymous_id,user_id,received_at,sent_at,timestamp,url,path,title,search,referrer,context_campaign_source,context_campaign_medium,context_campaign_name,context_campaign_term,context_campaign_content,context_ip,context_user_agent,context_campaign_type
ajs-9527e97fc714abe02a23b3d24cf09a36,507f191e810c19729de860ea,97980cfea0067,2015-02-23 22:28:55,2015-02-23 22:28:55,2015-02-23 22:28:55,https://www.getdbt.com/,/product,Product,?name=ferret,https://www.google.com/,facebook,social,autumn_collection_2015,foo,bar,8.8.8.8,"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36","typein"