Support BigQuery Materialized Views

[ran-eh]

Describe the feature

Support https://cloud.google.com/bigquery/docs/materialized-views-intro

Describe alternatives you've considered

1. add BQ specific configuration:

{{ config(
    materialized='view',
    materialized-view='true' # default 'false
)}}

2. create new materialization, initially for BQ only, over time perhaps implement for additional databases

{{ config(
    materialized='materialized-view',
)}}

Additional context

Some powerful stuff here https://cloud.google.com/bigquery/docs/materialized-views-intro

Who will this benefit?

BQ users

[drewbanin]

See also dbt-labs/dbt-core#1162

I think it makes sense to have one issue per database to track the addition of materialized views in dbt. Thanks for opening this one @ran-eh!

[sungchun12]

I highly recommend prioritizing this feature as it minimizes the need for extra incremental model logic for downstream datasets!

[drewbanin]

Right on @sungchun12! We're excited about materialized views too!

Our plan for supporting MVs is going to be to build them in a package. This will let us get some early feedback and iterate on their implementation without waiting for the regular dbt release cadence (every ~6 weeks or so).

This is pre-alpha, but if you want to check out the current approach and progress, you can have a look over here: https://github.com/fishtown-analytics/dbt-labs-experimental-features

[tuanchris]

@drewbanin Hi Drew, any update on this?

[drewbanin]

@jtcohen6 what do you think - should dbt support materialized views natively?

@tuanchris for what it's worth, there is a working materialization for BQ materialized views here: https://github.com/dbt-labs/dbt-labs-experimental-features/tree/master/materialized-views/macros/bigquery . You can either add this package to your packages.yml file, or copy/paste this materialization and supporting macro code directly into the macros/ directory of your dbt project.

[slagathorr]

Just a note, this is a pretty important feature to have, especially as folks are trying to build data objects in BQ to support analytics use cases, which MVs are pretty critical for in many cases.

[ian-wesley]

This would be very helpful for my current use case. Any update on when this is coming?

[var-bolt]

+1

[NoamMosko]

+1 :)

[sboyaci]

+1

[THEdavehogue]

+1

[enzorooo]

+1

[kieronellis]

+1

[pmunari]

+1

[araa47]

+1

[ytaniguchi8]

+1 :)

[Bruno1298]

+1

[borislitvak]

+1

[buckybush]

+1

[gscs-data]

+2

[octavio-melo]

+1

[bilus]

+1

[dinigo]

+1

[willbryant]

Will you guys knock it off? The way to vote on things is to use the 👍 reaction on the description at the top, not to post comments.

[dinigo]

@willbryant : Upvotes won't prevent the issue from going "stale" and being automatically archived by the bot (which we definitely don't want).

Also upvotes wont bump the issue, so developers will not have a sense of priority.

But you can customise your subscription to prevent "alarm fatigue" (spam)

[marcoruggine]

+1, this is essential

[SPTKL]

+1

[darthhash]

+1

[ee-jayson]

• This is so needed!

[minhajpasha]

+1

[dylanbhughes]

+1

[dreinon]

+1

[yuganghu]

+1

[artem110101]

+1

[techadmin-getsales]

+1

[jmarbach]

+1

[kverburg]

+1

[geerdink]

+1

[minghanxchiang]

+1

[dinigo]

This was implemented in dbt v1.7 dbt-labs/dbt-core#8449

Here an article from their blog
https://docs.getdbt.com/blog/announcing-materialized-views