Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand tests for BigQuery external tables #56

Open
2 tasks
Tracked by #260
jtcohen6 opened this issue Nov 29, 2020 · 1 comment
Open
2 tasks
Tracked by #260

Expand tests for BigQuery external tables #56

jtcohen6 opened this issue Nov 29, 2020 · 1 comment
Labels

Comments

@jtcohen6
Copy link
Collaborator

(split off from #41)

  • Figure out what's up with the ndjson source. Although staging the external table works well enough, I see this error as soon as I try to to query it: error message: JSON parsing error in row starting at position 0: Expected key. For the sake of merging BigQuery external tables #49, I've just commented out these tests.
  • Will BQ add DDL support for other external sources (Cloud Bigtable, Google Drive, Cloud SQL)?
@jtcohen6 jtcohen6 added enhancement New feature or request bigquery labels Nov 29, 2020
@jtcohen6 jtcohen6 mentioned this issue Nov 29, 2020
1 task
@drewmcdonald
Copy link

@jtcohen6 per your second point, yes! (full docs)

I was just doing some testing and got a sheets-backed table up and running with dbt-external-tables v0.6.2.

I haven't tried relying on schema inference or any other bells and whistles, but the stripped-down spec below works out of the box, which is really exciting!

YAML
version: 2

sources:
  - name: sheets
    database: my_project
    schema: my_dataset
  
    tables:
      - name: my_table
        external:
          location: "https://docs.google.com/spreadsheets/d/$BLAHBLAHBLAH"
          options:
            format: GOOGLE_SHEETS
            skip_leading_rows: 1

        columns:
          - name: thing_1
            data_type: STRING
          - name: thing_2
            data_type: INT64

@dataders dataders mentioned this issue Apr 4, 2024
@dataders dataders added this to the Upgrade CI 2.0 milestone Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants