Support BigQuery external tables

[yu-iskw]

Describe the feature

I want to use BigQuery external data sources feature with dbt.

Describe alternatives you've considered

I have no idea.But, according to the official documents, there is no way to manipulate external resources with only SQL statements.

• https://cloud.google.com/bigquery/external-data-sources

Additional context

My request is limited to only BigQuery.

Who will this benefit?

I am guessing many users uses BigQuery external data sources, including Google Spreadsheet, GCS and so on. By supporting the feature, they can get a great benefit to manage more BigQuery resources with dbt.

[jtcohen6]

there is no way to manipulate external resources with only SQL statements.

This is the thing stopping us from supporting BigQuery external tables in this package. The only ways forward I see are:

• BQ team adds DDL support for creating external tables
• We or a community member contribute a python external config adapter method to dbt-bigquery (docs), stackoverflow), which we could then leverage in this package.

If the latter is something you'd be interested in working on, I'm happy to try to help!

[yu-iskw]

@jtcohen6 Thank you for sharing it. BigQuery team may support the DDL of external data sources in the near future, as a result of filing the issues to them. If it is supported, we can move forward. I would love to contribute to it.

[jtcohen6]

https://cloud.google.com/blog/topics/developers-practitioners/smile-new-user-friendly-sql-capabilities-bigquery
https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_external_table_statement
the future is near indeed :))

[yu-iskw]

Yeah, they are GA now!
https://cloud.google.com/bigquery/docs/release-notes#October_14_2020

[jtcohen6]

@yu-iskw I'd be curious to get your eyes on the initial implementation in #49

[yu-iskw]

@jtcohen6 Super-duper! I will look into the PR. Thank you very much for the PR.