Show code problems found by workflow linter in the migration dashboard (

#1741)

## Changes
Show the code problems found by the experimental workflow linter
workflow in the migration dashboard.

### Linked issues
Resolves #1539 

### Functionality 

- [ ] added relevant user documentation
- [ ] added new CLI command
- [ ] modified existing command: `databricks labs ucx ...`
- [ ] added a new workflow
- [x] modified existing workflow: `experimental-workflow-linter`
- [ ] added a new table
- [ ] modified existing table: `...`

### Tests
<!-- How is this tested? Please see the checklist below and also
describe any other relevant tests -->

- [ ] manually tested
- [ ] added unit tests
- [ ] added integration tests
- [ ] verified on staging environment (screenshot attached)

---------

Co-authored-by: Serge Smertin <[email protected]>

README.md

@@ -39,6 +39,7 @@ See [contributing instructions](CONTRIBUTING.md) to help improve this project.
     * [Dependency CLI commands](#dependency-cli-commands)
     * [Table Migration Workflow Tasks](#table-migration-workflow-tasks)
     * [Other considerations](#other-considerations)
+  * [Jobs Static Code Analysis Workflow](#jobs-static-code-analysis-workflow)
 * [Utility commands](#utility-commands)
   * [`logs` command](#logs-command)
   * [`ensure-assessment-run` command](#ensure-assessment-run-command)
@@ -469,6 +470,17 @@ There are 3 main table migration workflows, targeting different table types. All
 
 [[back to top](#databricks-labs-ucx)]
 
+## Jobs Static Code Analysis Workflow
+
+> Please note that this is an experimental workflow.
+
+The `experimental-workflow-linter` workflow lints accessible code belonging to all workflows/jobs present in the
+workspace. The linting emits problems indicating what to resolve for making the code Unity Catalog compatible.
+
+![code compatibility problems](docs/code_compatibility_problems.png)
+
+[[back to top](#databricks-labs-ucx)]
+
 # Utility commands
 
 ## `logs` command

src/databricks/labs/ucx/queries/migration/main/02_0_code_compatibility_problems.md

@@ -0,0 +1,11 @@
+-- widget title=Workflow migration problems, row=2, col=0, size_x=2, size_y=8
+
+## 2 - Code compatibility problems
+
+The table on the right assist with verifying if workflows are Unity Catalog compatible. It can be filtered on the path,
+problem code and workflow name. The table:
+- Points to a problem detected in the code using the code path, workflow & task reference and start/end line & column;
+- Explains the problem with a human-readable message and a code.
+
+The code compatibility problems are updated after running
+[Jobs Static Code Analysis Workflow](https://github.com/databrickslabs/ucx/blob/main/README.md#workflow-linter-workflow).

src/databricks/labs/ucx/queries/migration/main/02_1_code_compatibility_problems.sql

@@ -0,0 +1,14 @@
+-- viz type=table, name=Workflow migration problems, search_by=path,code,job_name, columns=path,code,message,workflow_id,workflow_name,task_key,start_line,start_col,end_line,end_col
+-- widget title=Workflow migration problems, row=2, col=2, size_x=4, size_y=8
+SELECT
+    path,
+    code,
+    message,
+    job_id AS workflow_id,
+    job_name AS workflow_name,
+    task_key,
+    start_line,
+    start_col,
+    end_line,
+    end_col
+FROM $inventory.workflow_problems

src/databricks/labs/ucx/source_code/workflows.py

@@ -11,3 +11,8 @@ def lint_all_workflows(self, ctx: RuntimeContext):
         """[EXPERIMENTAL] Analyses all jobs for source code compatibility problems. This is an experimental feature,
         that is not yet fully supported."""
         ctx.workflow_linter.refresh_report(ctx.sql_backend, ctx.inventory_database)
+
+    @job_task(dashboard="migration_main", depends_on=[lint_all_workflows])
+    def migration_report(self, ctx: RuntimeContext):
+        """Refreshes the migration dashboard after all previous tasks have been completed. Note that you can access the
+        dashboard _before_ all tasks have been completed, but then only already completed information is shown."""