v1.9.1a1

Bug Fixes

• Fix import error in dbt bigquery adapter mock for dbt-bigquery<1.8 for ExecutionMode.AIRFLOW_ASYNC by @pankajkoti in #1548

Cut from: #1548 (branch: fix-dbt-bigquery-mock)

v1.9.0

Breaking changes

• When using LoadMode.DBT_LS, Cosmos will now attempt to use the dbtRunner as opposed to subprocess to run dbt ls.
  While this represents significant performance improvements (half the vCPU usage and some memory consumption improvement), this may not work in
  scenarios where users had multiple Python virtual environments to manage different versions of dbt and its adaptors. In those cases,
  please, set RenderConfig(invocation_mode=InvocationMode.SUBPROCESS) to have the same behaviour Cosmos had in previous versions.
  Additional information here <https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls>_ and here <https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#how-to-run-dbt-ls-invocation-mode>_.

Features

• Use dbtRunner in the DAG Processor when using LoadMode.DBT_LS if dbt-core is available by @tatiana in #1484. Additional information here <https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls>_.
• Allow users to opt-out of dbtRunner during DAG parsing with InvocationMode.SUBPROCESS by @tatiana in #1495. Check out the documentation <https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#how-to-run-dbt-ls-invocation-mode>_.
• Add structure to support multiple db for async operator execution by @pankajastro in #1483
• Support overriding the profile_config per dbt node or folder using config by @tatiana in #1492. More information here <https://astronomer.github.io/astronomer-cosmos/profiles/#profile-customise-per-node>_.
• Create and run accurate SQL statements when using ExecutionMode.AIRFLOW_ASYNC by @pankajkoti, @tatiana and @pankajastro in #1474
• Add AWS ECS task run execution mode by @CarlosGitto and @aoelvp94 in #1507
• Add support for running DbtSourceOperator individually by @victormacaubas in #1510
• Add setup task for async executions by @pankajastro in #1518
• Add teardown task for async executions by @pankajastro in #1529
• Add ProjectConfig.install_dbt_deps & change operator install_deps=True as default by @tatiana in #1521
• Extend Virtualenv operator and mock dbt adapters for setup & teardown tasks in ExecutionMode.AIRFLOW_ASYNC by @pankajkoti, @tatiana and @pankajastro in #1544

Bug Fixes

• Fix select complex intersection of three tag-based graph selectors by @tatiana in #1466
• Fix custom selector behaviour when the model name contains periods by @yakovlevvs and @60098727 in #1499
• Filter dbt and non-dbt kwargs correctly for async operator by @pankajastro in #1526

Enhancement

• Fix OpenLineage deprecation warning by @CorsettiS in #1449
• Move DbtRunner related functions into dbt/runner.py module by @tatiana in #1480
• Add on_warning_callback to DbtSourceKubernetesOperator and refactor previous operators by @LuigiCerone in #1501
• Gracefully error when users set incompatible RenderConfig.dbt_deps and operator_args install_deps by @tatiana in #1505
• Store compiled SQL as template field for ExecutionMode.AIRFLOW_ASYNC by @pankajkoti in #1534

Docs

• Improve RenderConfig arguments documentation by @tatiana in #1514
• Improve callback documentation by @tatiana in #1516
• Improve partial parsing docs by @tatiana in #1520
• Fix typo in selecting & excluding docs by @pankajastro in #1523
• Document async_py_requirements added in ExecutionConfig for ExecutionMode.AIRFLOW_ASYNC by @pankajkoti in #1545

Others

• Ignore dbt package tests when running Cosmos tests by @tatiana in #1502
• Refactor to consolidate async dbt adapter code by @pankajkoti in #1509
• Log elapsed time for sql file(s) upload/download by @pankajastro in #1536
• Remove the fallback operator for async task by @pankajastro in #1538
• GitHub Actions Dependabot: #1487
• Pre-commit updates: #1473, #1493, #1503, #1531

v1.9.0a6

(More work to come in the stable 1.9 release)

Breaking changes

• When using LoadMode.DBT_LS, Cosmos will now attempt to use the dbtRunner as opposed to subprocess to run dbt ls. While this represents significant performance improvements (half the vCPU usage and some memory consumption improvement), this may not work in scenarios where users had multiple Python virtual environments to manage different versions of dbt and its adaptors. In those cases, please set RenderConfig(invocation_mode=InvocationMode.SUBPROCESS) to have the same behaviour Cosmos had in previous versions. Additional information here and here.

Features

• Use dbtRunner in the DAG Processor when using LoadMode.DBT_LS if dbt-core is available by @tatiana in #1484. Additional information here.
• Allow users to opt-out of dbtRunner during DAG parsing with InvocationMode.SUBPROCESS by @tatiana in #1495. Check out the documentation.
• Add structure to support multiple db for async operator execution by @pankajastro in #1483
• Support overriding the profile_config per dbt node or folder using config by @tatiana in #1492. More information here.
• Use dbt to generate the full SQL and support different materializations for BQ for ExecutionMode.AIRFLOW_ASYNC by @pankajkoti in #1474
• Add setup task for async executions in #1518 by @pankajastro

Bug Fixes

• Fix select complex intersection of three tag-based graph selectors by @tatiana in #1466

Enhancement

• Fix OpenLineage deprecation warning by @CorsettiS in #1449
• Move DbtRunner related functions into dbt/runner.py module by @tatiana in #1480
• Add on_warning_callback to DbtSourceKubernetesOperator and refactor previous operators by @LuigiCerone in #1501

Others

• Ignore dbt package tests when running Cosmos tests by @tatiana in #1502
• GitHub Actions Dependabot: #1487
• Pre-commit updates: #1473, #1493

Cut from: main after merging #1492

v1.9.0a5

(More work to come in the stable 1.9 release)

Breaking changes

• When using LoadMode.DBT_LS, Cosmos will now attempt to use the dbtRunner as opposed to subprocess to run dbt ls. While this represents significant performance improvements (half the vCPU usage and some memory consumption improvement), this may not work in scenarios where users had multiple Python virtual environments to manage different versions of dbt and its adaptors. In those cases, please set RenderConfig(invocation_mode=InvocationMode.SUBPROCESS) to have the same behaviour Cosmos had in previous versions. Additional information here and here.

Features

• Use dbtRunner in the DAG Processor when using LoadMode.DBT_LS if dbt-core is available by @tatiana in #1484. Additional information here.
• Allow users to opt-out of dbtRunner during DAG parsing with InvocationMode.SUBPROCESS by @tatiana in #1495. Check out the documentation.
• Add structure to support multiple db for async operator execution by @pankajastro in #1483
• Support overriding the profile_config per dbt node or folder using config by @tatiana in #1492. More information here.
• Use dbt to generate the full SQL and support different materializations for BQ for ExecutionMode.AIRFLOW_ASYNC by @pankajkoti in #1474

Bug Fixes

• Fix select complex intersection of three tag-based graph selectors by @tatiana in #1466

Enhancement

• Fix OpenLineage deprecation warning by @CorsettiS in #1449
• Move DbtRunner related functions into dbt/runner.py module by @tatiana in #1480
• Add on_warning_callback to DbtSourceKubernetesOperator and refactor previous operators by @LuigiCerone in #1501

Others

• Ignore dbt package tests when running Cosmos tests by @tatiana in #1502
• GitHub Actions Dependabot: #1487
• Pre-commit updates: #1473, #1493

Cut from: main after merging #1492

v1.9.0a4

(More work to come in the stable 1.9 release)

Breaking changes

• When using LoadMode.DBT_LS, Cosmos will now attempt to use the dbtRunner as opposed to subprocess to run dbt ls. While this represents significant performance improvements (half the vCPU usage and some memory consumption improvement), this may not work in scenarios where users had multiple Python virtual environments to manage different versions of dbt and its adaptors. In those cases, please set RenderConfig(invocation_mode=InvocationMode.SUBPROCESS) to have the same behaviour Cosmos had in previous versions. Additional information here and here.

Features

• Use dbtRunner in the DAG Processor when using LoadMode.DBT_LS if dbt-core is available by @tatiana in #1484. Additional information here.
• Allow users to opt-out of dbtRunner during DAG parsing with InvocationMode.SUBPROCESS by @tatiana in #1495. Check out the documentation.
• Add structure to support multiple db for async operator execution by @pankajastro in #1483
• Support overriding the profile_config per dbt node or folder using config by @tatiana in #1492. More information here.

Bug Fixes

• Fix select complex intersection of three tag-based graph selectors by @tatiana in #1466

Enhancement

• Fix OpenLineage deprecation warning by @CorsettiS in #1449
• Move DbtRunner related functions into dbt/runner.py module by @tatiana in #1480

Others

• GitHub Actions Dependabot: #1487
• Pre-commit updates: #1473, #1493

Cut from: main after merging #1492

v1.9.0a3

(More work to come in the stable 1.9 release)

Features

• Add structure to support multiple db for async operator execution by @pankajastro in #1483
• Use dbtRunner in the DAG Processor when using LoadMode.DBT_LS if dbt-core is available by @tatiana in #1484
• Support overriding the profile_config per dbt node or folder using config by @tatiana in #1492

Bug Fixes

• Fix select complex intersection of three tag-based graph selectors by @tatiana in #1466

Enhancement

• Fix OpenLineage deprecation warning by @CorsettiS in #1449
• Support overriding the profile_name per node or folder using config by @tatiana in #1492
• Move DbtRunner related functions into dbt/runner.py module by @tatiana in #1480

Others

• GitHub Actions Dependabot: #1487
• Pre-commit updates: #1473, #1493

Cut from: #1492 (branch: override_profile_name_by_task)

v1.9.0a2

(More work to come in the stable 1.9 release)

Features

• Add structure to support multiple db for async operator execution by @pankajastro in #1483

Bug Fixes

• Fix select complex intersection of three tag-based graph selectors by @tatiana in #1466

Enhancement

• Fix OpenLineage deprecation warning by @CorsettiS in #1449
• Support overriding the profile_name per node or folder using config by @tatiana in #1492
• Move DbtRunner related functions into dbt/runner.py module by @tatiana in #1480

Others

• GitHub Actions Dependabot: #1487
• Pre-commit updates: #1473

Cut from: #1492

v1.9.0a1

(more to come soon)

Bug Fixes

• Fix select complex intersection of three tag-based graph selectors by @tatiana in #1466

Observations

This release was cut from the PR in progress #1466

v1.8.2

Bug Fixes

• Fix httpx.get exception handling while emitting telemetry by @tatiana in #1439
• Fix (not) rendering detached tests in TestBehavior.NONE and AFTER_ALL by @tatiana in #1463
• Fix detached test tasks names so they do not exceed 250 chars by @tatiana in #1464

Enhancement

• Allow users to opt-in or out (default) of detached test nodes by @tatiana in #1470. Learn more about this here.

Docs

• Docs: Fix broken links and rendering by @pankajastro in #1437
• Update operator args docs to include install_deps by @tatiana in #1456
• Improve Cosmos select docs to include latest graph operator support by @tatiana in #1467

Others

• Upgrade GitHub action artifacts upload-artifact & download-artifact to v4 by @pankajkoti in #1445
• Enable Depandabot to scan outdated Github Actions dependencies by @tatiana in #1446
• Pre-commit hook updates in #1459, #1441
• Dependabot Github action updates in #1451, #1452, #1453, #1454, #1455

v1.8.1

Bug Fixes

• Fix rendering dbt tests with multiple parents by @tatiana in #1433
• Add kwargs param in DocsOperator method upload_to_cloud_storage by @pankajastro in #1422

Docs

• Improve OpenLineage documentation by @tatiana in #1431

Others

• Enable Docs DAG in CI leveraging existing CI connections by @pankajkoti in #1428
• Install providers with airflow by @pankajkoti in #1432
• Remove unused docs dependency by @pankajastro in #1414
• Pre-commit hook updates in #1424

Full Changelog: astronomer-cosmos-v1.8.0...astronomer-cosmos-v1.8.1