BUG: NotFoundException from to_gbq when replacing table in project different from project_id parameter

[wnojopra]

Environment details

• Programming language: python3
• OS: Linux
• Language runtime version: 3.9.2
• Package version: 0.15.0

Steps to reproduce

Code Sample, a copy-pastable example

import pandas as pd

df = pd.DataFrame(
  {
    "A": pd.Series(1, index=list(range(2)))
  }
)

# 0) Start with an empty test_dataset. test_table does not exist at this point.


# 1) This successfully creates a table named test_table in dataset test_dataset in project willyn_data_project
df.to_gbq('willyn_data_project.test_dataset.test_table', project_id='willyn_job_project')

# 2) This fails with pandas_gbq.gbq.NotFoundException: Table does not exist
df.to_gbq('willyn_data_project.test_dataset.test_table', project_id='willyn_job_project', if_exists='replace')

Problem description

I am working on a project where researchers can write to a dataset in one GCP project, but must run query jobs in another. In the above code example, willyn_data_project is the name of the project that contains the dataset, and willyn_job_project is the name of the project where query jobs are executed.

When attempting to use if_exists='replace', we get the error pandas_gbq.gbq.NotFoundException: Table does not exist. I believe what is happening is we are attempting to delete a table from willyn_job_project, when we should be deleting a table in willyn_data_project.

My best guess would be this line in delete_and_recreate_table needs to be updated to use the correct project.

Expected Output

No error.

[tswast]

Thanks for the report! I think you are right in that the wrong project ID is being used.