Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create an Airflow DAG/Task Group from a dbt selector argument #67

Closed
chrishronek opened this issue Dec 22, 2022 · 5 comments
Closed

Create an Airflow DAG/Task Group from a dbt selector argument #67

chrishronek opened this issue Dec 22, 2022 · 5 comments
Assignees
@chrishronek
Labels
customer request An Astronomer customer made requested this enhancement New feature or request

Comments

@chrishronek
Copy link
Contributor

chrishronek commented Dec 22, 2022 •
edited
Loading

If this were completed, the DbtTaskGroup and DbtDag parsers would be able to parse given a selector argument like so:

DbtTaskGroup(project="...", selector="tag:foo+", ...)
@chrishronek chrishronek added enhancement New feature or request customer request An Astronomer customer made requested this labels Dec 22, 2022
@chrishronek chrishronek self-assigned this Dec 22, 2022
@chrishronek
Copy link
Contributor Author

chrishronek commented Dec 22, 2022 •
edited
Loading

See dbt docs on selection here

5 Selection Methods:

  1. Path

  2. Tag. There are 3 ways to set a tag, as listed in the description:

    • Project file
    • Config Property
    • Config block

  3. Config

  4. Test Type

  5. Test Name

@chrishronek chrishronek linked a pull request Dec 23, 2022 that will close this issue
Include model tags in parsers as well as a list parameter to filter them #69
Closed
3 tasks
@dweaver33
Copy link

We will need this as well.

@jlaneve jlaneve mentioned this issue Jan 9, 2023
Merged
jlaneve added a commit that referenced this issue Jan 10, 2023
@jlaneve @chrishronek
Refactor to support new features and code maintainability (#77)
0abc67d 
This is a pretty large refactor of how we were doing parsing and
generation to support a few goals:
- General code maintainability and extensibility
- Subclass Airflow's DAG and TaskGroup model instead of returning them
- Remove `pydantic` dependency in favor of dataclasses (closes #74)
- Add different testing types (`after_each`, `after_all`, `none`)
(closes #62)
- Add tags support (from code and from yml config) (partially addresses
#67)
- Add docs for testing types and tags

Co-authored-by: Chris Hronek <[email protected]>
@jlaneve
Copy link
Collaborator

jlaneve commented Jan 10, 2023

@pohek321 want to split these into different issues for each of the selector types?

@chrishronek
Copy link
Contributor Author

  • Created issue 89 for path selector method
  • Created issue 90 for config selector method
  • Created issue 91 for test_type selector method
  • Created issue 92 for test_name selector method

@chrishronek
Copy link
Contributor Author

Closing issue since we split it out into smaller issues.

@chrishronek chrishronek closed this as not planned Won't fix, can't repro, duplicate, stale Jan 23, 2023
@dosubot dosubot bot mentioned this issue Nov 28, 2023
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chrishronek chrishronek

Labels
customer request An Astronomer customer made requested this enhancement New feature or request
Projects
None yet
Milestone
No milestone
3 participants
@jlaneve @dweaver33 @chrishronek