Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dbt seed fails when multiple schemas configured #516

Open
jausanca opened this issue Feb 11, 2025 · 0 comments · May be fixed by #517
Open

dbt seed fails when multiple schemas configured #516

jausanca opened this issue Feb 11, 2025 · 0 comments · May be fixed by #517
Labels
bug Something isn't working

Comments

@jausanca
Copy link
Contributor

Describe the bug

dbt seed fails when multiple schemas are configured where not all af them contain seed nodes. For each target database not containing seeds, it prints the following error:

Glue adapter: An error occurred (EntityNotFoundException) when calling the GetTables operation: Database <database name> not found.

Steps To Reproduce

Execute dbt seed command on a project where not all used schemas contain seed nodes.

Expected behavior

Selected seed nodes are pushed as tables to the glue catalog.

Screenshots and log output

10:39:08  Glue adapter: An error occurred (EntityNotFoundException) when calling the GetTables operation: Database <database name> not found.
10:39:08  
10:39:08  Finished running  in 0 hours 0 minutes and 1.75 seconds (1.75s).
10:39:08  Encountered an error:
'NoneType' object is not iterable
10:39:08  Traceback (most recent call last):
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/cli/requires.py", line 153, in wrapper
    result, success = func(*args, **kwargs)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/cli/requires.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/cli/requires.py", line 235, in wrapper
    return func(*args, **kwargs)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/cli/requires.py", line 264, in wrapper
    return func(*args, **kwargs)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/cli/requires.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/cli/requires.py", line 328, in wrapper
    return func(*args, **kwargs)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/cli/main.py", line 710, in seed
    results = task.run()
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/task/runnable.py", line 588, in run
    result = self.execute_with_hooks(selected_uids)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/task/runnable.py", line 526, in execute_with_hooks
    before_run_status = self.before_run(adapter, selected_uids)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/task/run.py", line 995, in before_run
    self.populate_adapter_cache(adapter, required_schemas)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/task/runnable.py", line 492, in populate_adapter_cache
    adapter.set_relations_cache(cachable_nodes)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/adapters/base/impl.py", line 572, in set_relations_cache
    self._relations_cache_for_schemas(relation_configs, required_schemas)
  File "/home/jsanjuan/samples/test_dbt_project/.venv/lib/python3.10/site-packages/dbt/adapters/base/impl.py", line 548, in _relations_cache_for_schemas
    for relation in future.result():
TypeError: 'NoneType' object is not iterable

System information

The output of dbt --version:

Core:
  - installed: 1.9.2
  - latest:    1.9.2 - Up to date!

Plugins:
  - spark: 1.9.0 - Update available!
  - glue:  1.9.0 - Up to date!

The operating system you're using:
Ubuntu 22.04

The output of python --version:
Python 3.10.12

Additional context

Root cause is related to list_relations_without_caching method from the adapter not returning appropiate result when listing relations of non existing databases.
@jausanca jausanca added the bug Something isn't working label Feb 11, 2025
@jausanca jausanca linked a pull request Feb 11, 2025 that will close this issue
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix list_relations_without_caching for missing database jausanca/dbt-glue
1 participant
@jausanca