Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure: test_table_migration_convert_manged_to_external #3453

Closed
github-actions bot opened this issue Dec 18, 2024 · 4 comments · Fixed by #3459
Closed

Test failure: test_table_migration_convert_manged_to_external #3453

github-actions bot opened this issue Dec 18, 2024 · 4 comments · Fixed by #3459
Assignees

Comments

@github-actions
Copy link

❌ test_table_migration_convert_manged_to_external: databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details. (4m37.335s)
databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details.
[gw8] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
05:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.eMy9/config.yml) doesn't exist.
05:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:27 INFO [databricks.labs.ucx.install] Fetching installations...
05:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.eMy9 is corrupted. Skipping...
05:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+720241218052751
05:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:28 INFO [databricks.labs.ucx.install] Creating dashboards...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.eMy9/README for the next steps.
05:28 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/200680041168966
05:28 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/200680041168966/runs/391439277667710
05:28 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/200680041168966
05:28 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.eMy9/config.yml) doesn't exist.
05:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:27 INFO [databricks.labs.ucx.install] Fetching installations...
05:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.eMy9 is corrupted. Skipping...
05:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+720241218052751
05:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:28 INFO [databricks.labs.ucx.install] Creating dashboards...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:28 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:28 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:28 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.eMy9/README for the next steps.
05:28 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/200680041168966
05:28 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/200680041168966/runs/391439277667710
05:28 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/200680041168966
05:28 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:31 INFO [databricks.labs.ucx.install] Deleting UCX v0.52.1+720241218052751 from https://DATABRICKS_HOST
05:31 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_soxno
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=206462099709409, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=625833828504535, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=923254099467214, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=356783163900589, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1029653297325497, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=236122217435452, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=200680041168966, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1004863312099187, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=756102937432835, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=698299497379257, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=426849429454101, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=169511700061292, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=285613694697081, as it is no longer needed
05:31 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:31 INFO [databricks.labs.ucx.install] Deleting secret scope
05:31 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw8] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #315

@github-actions github-actions bot added the bug label Dec 18, 2024
Copy link
Author

❌ test_table_migration_convert_manged_to_external: databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details. (5m19.415s)
databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details.
[gw6] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
05:31 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.awgz/config.yml) doesn't exist.
05:31 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:31 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:31 INFO [databricks.labs.ucx.install] Fetching installations...
05:31 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.awgz is corrupted. Skipping...
05:31 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:31 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:31 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:31 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+720241219053121
05:31 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:31 INFO [databricks.labs.ucx.install] Creating dashboards...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:32 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.awgz/README for the next steps.
05:32 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/470550384133407
05:32 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/470550384133407/runs/670815175753579
05:32 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/470550384133407
05:32 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:31 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.awgz/config.yml) doesn't exist.
05:31 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:31 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:31 INFO [databricks.labs.ucx.install] Fetching installations...
05:31 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.awgz is corrupted. Skipping...
05:31 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:31 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:31 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:31 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+720241219053121
05:31 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:31 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:31 INFO [databricks.labs.ucx.install] Creating dashboards...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:31 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:31 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:31 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:32 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.awgz/README for the next steps.
05:32 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/470550384133407
05:32 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/470550384133407/runs/670815175753579
05:32 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/470550384133407
05:32 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:35 INFO [databricks.labs.ucx.install] Deleting UCX v0.52.1+720241219053121 from https://DATABRICKS_HOST
05:35 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_shnsi
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1073599905029090, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=323566237536617, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=285885623357525, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=609331002775999, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=205989018722379, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=927357795921424, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=364170123834355, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=27685712979281, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=67823803881095, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=926368570877135, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1054281486141817, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=470550384133407, as it is no longer needed
05:35 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=51413244542458, as it is no longer needed
05:36 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:36 INFO [databricks.labs.ucx.install] Deleting secret scope
05:36 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw6] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #316

@JCZuurmond JCZuurmond self-assigned this Dec 19, 2024
Copy link
Author

❌ test_table_migration_convert_manged_to_external: databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details. (4m36.133s)
databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details.
[gw2] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
14:47 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.ctMh/config.yml) doesn't exist.
14:47 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
14:47 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
14:47 INFO [databricks.labs.ucx.install] Fetching installations...
14:47 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.ctMh is corrupted. Skipping...
14:47 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
14:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:47 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+820241219144736
14:47 INFO [databricks.labs.ucx.install] Creating ucx schemas...
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
14:47 INFO [databricks.labs.ucx.install] Creating dashboards...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.ctMh/README for the next steps.
14:47 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/698953927997122
14:47 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/698953927997122/runs/889500275817694
14:47 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/698953927997122
14:47 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:47 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.ctMh/config.yml) doesn't exist.
14:47 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
14:47 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
14:47 INFO [databricks.labs.ucx.install] Fetching installations...
14:47 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.ctMh is corrupted. Skipping...
14:47 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
14:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:47 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+820241219144736
14:47 INFO [databricks.labs.ucx.install] Creating ucx schemas...
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
14:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
14:47 INFO [databricks.labs.ucx.install] Creating dashboards...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
14:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
14:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:47 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.ctMh/README for the next steps.
14:47 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/698953927997122
14:47 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/698953927997122/runs/889500275817694
14:47 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/698953927997122
14:47 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:51 INFO [databricks.labs.ucx.install] Deleting UCX v0.52.1+820241219144736 from https://DATABRICKS_HOST
14:51 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_slgsv
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=280620007822740, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=988597633750175, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=826145918606091, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=239137178040238, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=508737009732864, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=583103931440063, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=698953927997122, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=341986989861811, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=250349745474707, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=667042113716167, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1026845816703659, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=602181557504532, as it is no longer needed
14:51 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=513134794092322, as it is no longer needed
14:51 INFO [databricks.labs.ucx.install] Deleting cluster policy
14:51 INFO [databricks.labs.ucx.install] Deleting secret scope
14:51 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw2] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #317

Copy link
Author

❌ test_table_migration_convert_manged_to_external: databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details. (4m46.173s)
databricks.sdk.errors.sdk.OperationFailed: failed to reach TERMINATED or SKIPPED, got RunLifeCycleState.INTERNAL_ERROR: Task parse_logs failed with message: Workload failed, see run output for details.
[gw8] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
05:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5XqB/config.yml) doesn't exist.
05:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:27 INFO [databricks.labs.ucx.install] Fetching installations...
05:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5XqB is corrupted. Skipping...
05:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+720241220052713
05:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:27 INFO [databricks.labs.ucx.install] Creating dashboards...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5XqB/README for the next steps.
05:27 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/483619794028551
05:27 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/483619794028551/runs/506450462099055
05:27 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/483619794028551
05:27 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5XqB/config.yml) doesn't exist.
05:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
05:27 INFO [databricks.labs.ucx.install] Fetching installations...
05:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5XqB is corrupted. Skipping...
05:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:27 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+720241220052713
05:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:27 INFO [databricks.labs.ucx.install] Creating dashboards...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:27 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:27 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:27 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.5XqB/README for the next steps.
05:27 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/483619794028551
05:27 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/483619794028551/runs/506450462099055
05:27 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/483619794028551
05:27 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:31 INFO [databricks.labs.ucx.install] Deleting UCX v0.52.1+720241220052713 from https://DATABRICKS_HOST
05:31 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_snk59
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=908504633652153, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=700602459927653, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=805079187682479, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=30270610204370, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=483619794028551, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=696869451969454, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=760561595038555, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=470216993776134, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=823052061458245, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=476093617799155, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=32917710540898, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=991636447349916, as it is no longer needed
05:31 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=346122746858003, as it is no longer needed
05:31 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:31 INFO [databricks.labs.ucx.install] Deleting secret scope
05:31 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw8] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #318

Copy link
Author

❌ test_table_migration_convert_manged_to_external: databricks.sdk.errors.platform.ResourceDoesNotExist: Run 629287316796175 does not exist. (2m43.858s)
databricks.sdk.errors.platform.ResourceDoesNotExist: Run 629287316796175 does not exist.
[gw2] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python
13:08 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.X7k9/config.yml) doesn't exist.
13:08 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
13:08 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
13:08 INFO [databricks.labs.ucx.install] Fetching installations...
13:08 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.X7k9 is corrupted. Skipping...
13:08 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
13:08 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:08 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:08 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+820241220130828
13:08 INFO [databricks.labs.ucx.install] Creating ucx schemas...
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
13:08 INFO [databricks.labs.ucx.install] Creating dashboards...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.X7k9/README for the next steps.
13:08 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/589587933241116
13:08 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/589587933241116/runs/629287316796175
13:08 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/589587933241116
13:08 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
13:08 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.X7k9/config.yml) doesn't exist.
13:08 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
13:08 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
13:08 INFO [databricks.labs.ucx.install] Fetching installations...
13:08 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.X7k9 is corrupted. Skipping...
13:08 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
13:08 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:08 DEBUG [tests.integration.conftest] Waiting for clusters to start...
13:08 INFO [databricks.labs.ucx.install] Installing UCX v0.52.1+820241220130828
13:08 INFO [databricks.labs.ucx.install] Creating ucx schemas...
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
13:08 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
13:08 INFO [databricks.labs.ucx.install] Creating dashboards...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
13:08 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
13:08 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
13:08 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.X7k9/README for the next steps.
13:08 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/589587933241116
13:08 INFO [databricks.labs.ucx.installer.workflows] Started migrate-tables job: https://DATABRICKS_HOST#job/589587933241116/runs/629287316796175
13:08 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/589587933241116
13:08 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
13:10 INFO [databricks.labs.ucx.install] Deleting UCX v0.52.1+820241220130828 from https://DATABRICKS_HOST
13:10 ERROR [databricks.labs.ucx.install] Check if /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.X7k9 is present
[gw2] linux -- Python 3.10.15 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #320

github-merge-queue bot pushed a commit that referenced this issue Dec 20, 2024
## Changes
Use LTS Databricks runtime version because the convert to external table
migration strategy fails on the latest runtime

> Note: This postpones the problem! Which is preferred for the next
release as users can experience this problem with the latest UCX
release. However, it does not solve the problem.

### Linked issues

Resolves #3453 (for now)
Reslves #3461

### Functionality

- [x] modified existing workflow: `migrate-tables`

### Tests

- [x] reused integration tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant