-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add databricks labs ucx migrate-tables
and a related workflow
#670
Labels
feat/migration-index
mapping of databases to catalog or potentially other databases
migrate/external
go/uc/upgrade SYNC EXTERNAL TABLES step
migrate/managed
go/uc/upgrade Upgrade Managed Tables and Jobs
Comments
|
This was referenced Dec 6, 2023
Closed
8 tasks
1 task
This was referenced Mar 8, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
feat/migration-index
mapping of databases to catalog or potentially other databases
migrate/external
go/uc/upgrade SYNC EXTERNAL TABLES step
migrate/managed
go/uc/upgrade Upgrade Managed Tables and Jobs
databricks labs ucx migrate-tables to apply table mapping from CSV file with the following columns: src_workspace_name,dst_catalog,src_schema,dst_schema,src_table,dst_table . Consider the following stages:
Initial assessment
We scan all tables, generate an inventory, get all perms, etc.
We provide a mapping file the customer can customize, but seed the UC table name with something like the workspace name or ID as the catalog.
Customer can choose to modify the mapping or not
Initial migration
We use the mapping file to sync tables, apply perms, and persist the mapping data in inventory
Repeated migration - this is probably a job that runs on schedule and the user should be instructed to configure accordingly
We use the mapping data in inventory to re-sync any tables, but not re-apply permissions as these probably changed.
Fine-grained:
EXTERNAL
tables from cloud storage accounts #333Upstream dependencies:
databricks labs ucx table-mapping
command #669Next quarter:
_migrate_table_create_ctas
tomigrate-tables
workflow #332SYNC
is supposed to be idempotent and intended to run multiple times. #319databricks labs ucx validate-table-locations
command #673The text was updated successfully, but these errors were encountered: