Render vars when parsing DbtModel and appending upstream_models

[dojinkimm]

Description

When parsing DbtModel if I have vars in sql file, I fail to get valid upstream models (vars were not rendered as expected). This is because when add a model to upstream_models only Const types are allowed and Concat types are skipped.

-- sql file
SELECT
    column_1,
	column_2,
FROM {{ ref('stg_users_'~ var('country_code')) }} # ex) `stg_users_us` is added to upstream_models

I passed operator_args and used vars from operator_args when adding upstream_models. if variable exist but it is not defined in vars it raises KeyError and skips as before. If correct value exist for vars it the value is added up to upstream_model

Related Issue(s)

closes #495

Breaking Change?

Checklist

• I have added tests that prove my fix is effective or that my feature works

[netlify]

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name	Link
🔨 Latest commit	d2dfec6
🔍 Latest deploy log	https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/64fedc032a8970000807f3de

[codecov]

Codecov Report

Patch coverage: 76.00% and project coverage change: -0.05% ⚠️

Comparison is base (74dff86) 91.82% compared to head (d2dfec6) 91.78%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #502      +/-   ##
==========================================
- Coverage   91.82%   91.78%   -0.05%     
==========================================
  Files          49       49              
  Lines        1896     1911      +15     
==========================================
+ Hits         1741     1754      +13     
- Misses        155      157       +2     

Files Changed	Coverage Δ
cosmos/converter.py	95.58% <ø> (ø)
cosmos/dbt/parser/project.py	90.47% <73.91%> (-0.39%)	⬇️
cosmos/__init__.py	100.00% <100.00%> (ø)
cosmos/dbt/graph.py	98.68% <100.00%> (+<0.01%)	⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tatiana]

Hi @dojinkimm , this is looking great. Thanks for the fix and the test! Once the tests pass after the rebase, I'll merge this change. We'll release it as part of the 1.1.1 release.

Out of curiosity, why are you using load_method=LoadMode.CUSTOM?

[dojinkimm]

Out of curiosity, why are you using load_method=LoadMode.CUSTOM?

@tatiana I am currently using astronomer-cosmos 0.6.8 due to some package conflicts in my environment, and load_method does not exist in that version. So, I was using default parsing logic

[tatiana]

Hi @dojinkimm , I see - is there any chance we could help you with the dependency conflict? Please feel free to log a ticket.
We were aiming to release this as part of 1.1.1 - and I'd love to see if we can help you move to the newer version.