workflow dependencies graph visualisation #33
Conversation
itayL1
force-pushed
the
add-workflow-graph-visualization
branch
30 times, most recently
from
dd9b2e6 to
51db884
Compare
itayL1
requested review from
AlonLek,
meoril,
moranbel,
mzedayda,
RabinovichSagi,
shanioren,
thegreymatter and
yaronmat
itayL1
force-pushed
the
add-workflow-graph-visualization
branch
from
51db884 to
f55846b
Compare
| return self._graph | ||
|
|
||
| def add_template_subgraph(self, dsl_wf_template: WorkflowTemplate, parent_node_id: Optional[str]): | ||
| wf_template = dsl_wf_template.to_model(embed_workflow_templates=False) |
There was a problem hiding this comment.
@mzedayda something weird happened here, when I changed to embed_workflow_templates=True, the tests still passed (the graph was built properly). does it make sense? do we still use templateRef s when embed_workflow_templates=True?
There was a problem hiding this comment.
my mistake, it doesn't work when embed_workflow_templates=True, as it should
itayL1
force-pushed
the
add-workflow-graph-visualization
branch
3 times, most recently
from
7c6d3b5 to
cf0b02c
Compare
| @@ -8,7 +8,7 @@ isort: | |||
| isort argo_workflow_tools tests examples | |||
|
|
|||
| test: | |||
| poetry run pytest --durations=5 tests/dsl | |||
| poetry run pytest --durations=5 tests/argo_workflow_tools/dsl/ | |||
There was a problem hiding this comment.
same as it was before. also, the tests in tests/argo_workflow_tools/client don't work.
| def add_template_subgraph(self, deps_graph: nx.DiGraph, dsl_wf_template: WorkflowTemplate, | ||
| parent_node_id: Optional[str]): | ||
| # use embed_workflow_templates=False so the workflow model will use workflowRef when using another | ||
| # workflow template. this is how this implementation identifies the decencies in the graph. |
| @@ -0,0 +1,21 @@ | |||
| [tool.poetry] | |||
There was a problem hiding this comment.
Kinda weird this is a separate package in the same repo no?
Are we sure we don't want to open a new repo for it?
There was a problem hiding this comment.
yes, it's a bit weird, but I don't think that it is worth another repo, which will be less convenient. I was actually surprised that poetry doesn't support multiple packages under the same project (here is a thread on it). WDYT?
|
In the current implementation, the graph only shows parent-child dependencies, is that what we want? wouldn't it be more informative to print the actual dag? |
|
@mzedayda Yes what I want is the high-level dependencies graph between the workflows. this is very useful at least in our project, where we use nest many workflows in the complicated release flows. also, plotting the real dag is very complex, there are a lot of edge cases that are resolved by argo at runtime (loops, condition etc.). |
…of nested workflow templates (in a new isolated package)
itayL1
force-pushed
the
add-workflow-graph-visualization
branch
from
cf0b02c to
eee6648
Compare
add support for the visualization of the workflow dependencies graph of nested workflow templates (in a new isolated package).
example usage:
for this template (from the test):
this graph was generated:
