Skip to content

Commit

Permalink
Address review comments.
Browse files Browse the repository at this point in the history
Signed-off-by: Revital Sur <[email protected]>
  • Loading branch information
revit13 committed Jan 27, 2025
1 parent c5117e5 commit 5bba22c
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 3 deletions.
2 changes: 1 addition & 1 deletion examples/kfp-pipelines/superworkflows/ray/kfp_v2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Another useful feature of the KFP v2 is the `Json` editor for the `dict` type in
- It creates just one run that includes all the nested transfroms and their sub-tasks.
- No need for additional component as `executeSubWorkflowComponent.yaml`. All the implementation in the same pipeline file.
- In superpipelines of KFP v1 there exists an option to override the common parameters with specific values for each one of the transforms. This option is missing in the KFP v2 superpipelines.
- In kfp V2 pipelines the user is requested to insert a unique string for the ray cluster created at run creation time (called `ray_run_id_KFPv2`). This is because in KFPv2 dsl.RUN_ID_PLACEHOLDER is deprecated and cannot be used since SDK 2.5.0.
- In kfp V2 pipelines the user is requested to insert a unique string for the ray cluster created at run creation time (called `ray_run_id_KFPv2`). This is because in KFPv2 `dsl.RUN_ID_PLACEHOLDER` is deprecated and cannot be used since SDK 2.5.0 and we cannot generate a unique string at run-time, see https://github.com/kubeflow/pipelines/issues/10187.

### How to compile the superpipeline
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,12 @@ def super_pipeline(
p1_pipeline_data_max_files: int = -1,
p1_pipeline_data_num_samples: int = -1,
p1_pipeline_data_checkpointing: bool = False,
p1_pipeline_ray_run_id_KFPv2: str = "",
# noop step parameters
p2_name: str = "noop",
p2_skip: bool = False,
p2_noop_sleep_sec: int = 10,
p2_ray_name: str = "noop-kfp-ray",
p2_ray_run_id_KFPv2: str = "",
p2_ray_head_options: dict = {"cpu": 1, "memory": 4, "image_pull_secret": "", "image": noop_image},
p2_ray_worker_options: dict = {
"replicas": 2,
Expand All @@ -76,7 +76,6 @@ def super_pipeline(
# Document ID step parameters
p3_name: str = "doc_id",
p3_ray_name: str = "docid-kfp-ray",
p3_ray_run_id_KFPv2: str = "",
p3_ray_head_options: dict = {"cpu": 1, "memory": 4, "image_pull_secret": "", "image": doc_id_image},
p3_ray_worker_options: dict = {
"replicas": 2,
Expand Down

0 comments on commit 5bba22c

Please sign in to comment.