diff --git a/README.md b/README.md index 113204a368..98376529cf 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,7 @@ See [contributing instructions](CONTRIBUTING.md) to help improve this project. * [Dependency CLI commands](#dependency-cli-commands) * [Table Migration Workflow Tasks](#table-migration-workflow-tasks) * [Other considerations](#other-considerations) + * [Jobs Static Code Analysis Workflow](#jobs-static-code-analysis-workflow) * [Utility commands](#utility-commands) * [`logs` command](#logs-command) * [`ensure-assessment-run` command](#ensure-assessment-run-command) @@ -469,6 +470,17 @@ There are 3 main table migration workflows, targeting different table types. All [[back to top](#databricks-labs-ucx)] +## Jobs Static Code Analysis Workflow + +> Please note that this is an experimental workflow. + +The `experimental-workflow-linter` workflow lints accessible code belonging to all workflows/jobs present in the +workspace. The linting emits problems indicating what to resolve for making the code Unity Catalog compatible. + +![code compatibility problems](docs/code_compatibility_problems.png) + +[[back to top](#databricks-labs-ucx)] + # Utility commands ## `logs` command diff --git a/docs/code_compatibility_problems.png b/docs/code_compatibility_problems.png new file mode 100644 index 0000000000..323c40199b Binary files /dev/null and b/docs/code_compatibility_problems.png differ diff --git a/src/databricks/labs/ucx/queries/migration/main/02_0_code_compatibility_problems.md b/src/databricks/labs/ucx/queries/migration/main/02_0_code_compatibility_problems.md new file mode 100644 index 0000000000..75e97b24b2 --- /dev/null +++ b/src/databricks/labs/ucx/queries/migration/main/02_0_code_compatibility_problems.md @@ -0,0 +1,11 @@ +-- widget title=Workflow migration problems, row=2, col=0, size_x=2, size_y=8 + +## 2 - Code compatibility problems + +The table on the right assist with verifying if workflows are Unity Catalog compatible. It can be filtered on the path, +problem code and workflow name. The table: +- Points to a problem detected in the code using the code path, workflow & task reference and start/end line & column; +- Explains the problem with a human-readable message and a code. + +The code compatibility problems are updated after running +[Jobs Static Code Analysis Workflow](https://github.com/databrickslabs/ucx/blob/main/README.md#workflow-linter-workflow). diff --git a/src/databricks/labs/ucx/queries/migration/main/02_1_code_compatibility_problems.sql b/src/databricks/labs/ucx/queries/migration/main/02_1_code_compatibility_problems.sql new file mode 100644 index 0000000000..70994baca7 --- /dev/null +++ b/src/databricks/labs/ucx/queries/migration/main/02_1_code_compatibility_problems.sql @@ -0,0 +1,14 @@ +-- viz type=table, name=Workflow migration problems, search_by=path,code,job_name, columns=path,code,message,workflow_id,workflow_name,task_key,start_line,start_col,end_line,end_col +-- widget title=Workflow migration problems, row=2, col=2, size_x=4, size_y=8 +SELECT + path, + code, + message, + job_id AS workflow_id, + job_name AS workflow_name, + task_key, + start_line, + start_col, + end_line, + end_col +FROM $inventory.workflow_problems \ No newline at end of file diff --git a/src/databricks/labs/ucx/source_code/workflows.py b/src/databricks/labs/ucx/source_code/workflows.py index c3775dfb1c..d58f2978aa 100644 --- a/src/databricks/labs/ucx/source_code/workflows.py +++ b/src/databricks/labs/ucx/source_code/workflows.py @@ -11,3 +11,8 @@ def lint_all_workflows(self, ctx: RuntimeContext): """[EXPERIMENTAL] Analyses all jobs for source code compatibility problems. This is an experimental feature, that is not yet fully supported.""" ctx.workflow_linter.refresh_report(ctx.sql_backend, ctx.inventory_database) + + @job_task(dashboard="migration_main", depends_on=[lint_all_workflows]) + def migration_report(self, ctx: RuntimeContext): + """Refreshes the migration dashboard after all previous tasks have been completed. Note that you can access the + dashboard _before_ all tasks have been completed, but then only already completed information is shown."""