From 10b055e8a9e34e59c8e11e3ced4919502b23a3f0 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Wed, 11 Dec 2019 08:19:04 +0100 Subject: [PATCH 1/6] lift requirement for all workflow inputs to be filled --- versions/development/SPEC.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/versions/development/SPEC.md b/versions/development/SPEC.md index 8315e11a..abc98872 100644 --- a/versions/development/SPEC.md +++ b/versions/development/SPEC.md @@ -2602,8 +2602,12 @@ In this example, `i`, and `f` are inputs to this task even though `i` is not dir Workflows have inputs that must be satisfied to run them, just like tasks. Inputs to the workflow are provided as a key/value map where the key is of the form `workflow_name.input_name`. -* If a workflow is to be used as a sub-workflow it must ensure that all of the inputs to its calls are satisfied. -* If a workflow will only ever be submitted as a top-level workflow, it may optionally leave its tasks' inputs unsatisfied. This then forces the engine to additionally supply those inputs at run time. In this case, the inputs' names must be qualified in the inputs as `workflow_name.task_name.input_name`. +* Workflows may optionally leave its tasks' required inputs unsatisfied. This then forces the engine to additionally supply those inputs at run time. In this case, the inputs' names must be qualified in the inputs as `workflow_name.task_name.input_name`. +* Optional inputs (or required inputs with defaults) for tasks are not required to be + satisfied. +* Inputs should be fully qualified. For example `workflow_name.workflow_input`, + `workflow_name.some_task.task_input`, `workflow_name.subworkflow_name.subwf_task.input_name` + ,`workflow_name.subworkflow_name.another_subworkflow.yet_another_sub_wf.subwf_input` etc. Any declaration that appears outside the `input` section is considered an intermediate value and **not** a workflow input. Any declaration can always be moved inside the `input` block to make it overridable. From dcbfafa07b229e53db8ac8fe5b2ff781d955bc75 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Wed, 11 Dec 2019 13:17:29 +0100 Subject: [PATCH 2/6] address @davycats review comments --- versions/development/SPEC.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/versions/development/SPEC.md b/versions/development/SPEC.md index abc98872..905a7e14 100644 --- a/versions/development/SPEC.md +++ b/versions/development/SPEC.md @@ -2602,12 +2602,14 @@ In this example, `i`, and `f` are inputs to this task even though `i` is not dir Workflows have inputs that must be satisfied to run them, just like tasks. Inputs to the workflow are provided as a key/value map where the key is of the form `workflow_name.input_name`. -* Workflows may optionally leave its tasks' required inputs unsatisfied. This then forces the engine to additionally supply those inputs at run time. In this case, the inputs' names must be qualified in the inputs as `workflow_name.task_name.input_name`. -* Optional inputs (or required inputs with defaults) for tasks are not required to be +* Workflows may optionally leave its calls' required inputs unsatisfied. This then forces the engine to additionally supply those inputs at run time. In this case, the inputs' names must be qualified in the inputs as `workflow_name.task_name.input_name`. +* Optional inputs (or required inputs with defaults) for calls are not required to be satisfied. * Inputs should be fully qualified. For example `workflow_name.workflow_input`, `workflow_name.some_task.task_input`, `workflow_name.subworkflow_name.subwf_task.input_name` ,`workflow_name.subworkflow_name.another_subworkflow.yet_another_sub_wf.subwf_input` etc. + For more information checkout + [the section on fully qualified names](#fully-qualified-names--namespaced-identifiers). Any declaration that appears outside the `input` section is considered an intermediate value and **not** a workflow input. Any declaration can always be moved inside the `input` block to make it overridable. From ecd270d7f515eee32a8342e73114147fa4140015 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Tue, 28 Jan 2020 08:19:17 +0100 Subject: [PATCH 3/6] encourage filling in required inputs for tasks --- versions/development/SPEC.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/versions/development/SPEC.md b/versions/development/SPEC.md index abc98872..9e0e8f9c 100644 --- a/versions/development/SPEC.md +++ b/versions/development/SPEC.md @@ -2602,12 +2602,19 @@ In this example, `i`, and `f` are inputs to this task even though `i` is not dir Workflows have inputs that must be satisfied to run them, just like tasks. Inputs to the workflow are provided as a key/value map where the key is of the form `workflow_name.input_name`. -* Workflows may optionally leave its tasks' required inputs unsatisfied. This then forces the engine to additionally supply those inputs at run time. In this case, the inputs' names must be qualified in the inputs as `workflow_name.task_name.input_name`. -* Optional inputs (or required inputs with defaults) for tasks are not required to be - satisfied. +Workflows call tasks (and subworkflows) that have inputs as well. These inputs can be filled +in by the calling workflow, or left empty for the user to fill in. The rules for defining inputs are as follows: + * Inputs should be fully qualified. For example `workflow_name.workflow_input`, `workflow_name.some_task.task_input`, `workflow_name.subworkflow_name.subwf_task.input_name` ,`workflow_name.subworkflow_name.another_subworkflow.yet_another_sub_wf.subwf_input` etc. +* *Optional* inputs (or required inputs with defaults) for tasks are not required to be + satisfied. +* It is good practice for workflows to provide the *required* inputs for its tasks. If this is + not done the workflow cannot be called as a subworkflow as the call `input:` section does + not allow namespaced inputs. Also, if a workflow does not fill its tasks' required options then these should be supplied by the user, while not being listed in the workflow `input` section. This can cause confusion and make the workflow harder to use. + Engines may optionally enforce the good behavior of supplying all tasks' required inputs by the + calling workflow. Any declaration that appears outside the `input` section is considered an intermediate value and **not** a workflow input. Any declaration can always be moved inside the `input` block to make it overridable. From 60fcf597ee4d41f45b60102d0363e46899c7ed7b Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Thu, 6 Feb 2020 08:38:51 +0100 Subject: [PATCH 4/6] Integrate cjllanwarne's proposal --- versions/development/SPEC.md | 42 ++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 18 deletions(-) diff --git a/versions/development/SPEC.md b/versions/development/SPEC.md index 44ef2f51..f0c4c92e 100644 --- a/versions/development/SPEC.md +++ b/versions/development/SPEC.md @@ -2600,24 +2600,30 @@ In this example, `i`, and `f` are inputs to this task even though `i` is not dir ## Computing Workflow Inputs -Workflows have inputs that must be satisfied to run them, just like tasks. Inputs to the workflow are provided as a key/value map where the key is of the form `workflow_name.input_name`. - -Workflows call tasks (and subworkflows) that have inputs as well. These inputs can be filled -in by the calling workflow, or left empty for the user to fill in. The rules for defining inputs are as follows: - -* Inputs should be fully qualified. For example `workflow_name.workflow_input`, - `workflow_name.some_task.task_input`, `workflow_name.subworkflow_name.subwf_task.input_name` - ,`workflow_name.subworkflow_name.another_subworkflow.yet_another_sub_wf.subwf_input` etc. - For more information checkout - [the section on fully qualified names](#fully-qualified-names--namespaced-identifiers). -* *Optional* inputs (or required inputs with defaults) for tasks are not required to be - satisfied. -* It is good practice for workflows to provide the *required* inputs for its tasks. If this is - not done the workflow cannot be called as a subworkflow as the call `input:` section does - not allow namespaced inputs. Also, if a workflow does not fill its tasks' required options then these should be supplied by the user, while not being listed in the workflow `input` section. This can cause confusion and make the workflow harder to use. - Engines may optionally enforce the good behavior of supplying all tasks' required inputs by the - calling workflow. - +Workflows have inputs that must be satisfied to run them, just like tasks. +Inputs to the workflow are provided as a key/value map where the key is of the +form `workflow_name.input_name`. + +* A task usually has its inputs supplied when called by a workflow. + * Example: `call my_task { input: my_task_input=... }` +* A workflow is allowd not to specify all inputs in this block. In this case, + the inputs bubble up to become an input to the workflow instaead. + * Example: an unsupplied input might have the fully-qualified name + `my_workflow.my_task.my_task_input.` + +* If that workflow is used as a subworkflow, the input is allowed to bubble up + again with a further-qualified name. + * Example: my_outer_workflow.my_workflow.my_task.my_task_input. +* There is currently no way to supply a bubbled-up input in an outer workflow's + call block. + * Example: one cannot say call my_workflow as subworkflow + `{ inputs: my_task.my_task_input=... }` +* All **required** inputs which bubble up like this must ultimately be provided + in the inputs set before the top-level workflow can be run. + * An engine must allow these optional bubbled-up inputs to be filled in by end + users. However, it is up to the engine how many layers of nesting it chooses + to expose in the UI. + Any declaration that appears outside the `input` section is considered an intermediate value and **not** a workflow input. Any declaration can always be moved inside the `input` block to make it overridable. Consider the following workflow: From 134f09035575e4cf7ff417f6a4e910f624deb432 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Thu, 6 Feb 2020 08:39:41 +0100 Subject: [PATCH 5/6] remove redundant new line --- versions/development/SPEC.md | 1 - 1 file changed, 1 deletion(-) diff --git a/versions/development/SPEC.md b/versions/development/SPEC.md index f0c4c92e..c040cfeb 100644 --- a/versions/development/SPEC.md +++ b/versions/development/SPEC.md @@ -2610,7 +2610,6 @@ form `workflow_name.input_name`. the inputs bubble up to become an input to the workflow instaead. * Example: an unsupplied input might have the fully-qualified name `my_workflow.my_task.my_task_input.` - * If that workflow is used as a subworkflow, the input is allowed to bubble up again with a further-qualified name. * Example: my_outer_workflow.my_workflow.my_task.my_task_input. From 549eed6b71b09f3fac23b30edca3e4828ac5a6e9 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Thu, 6 Feb 2020 08:44:27 +0100 Subject: [PATCH 6/6] better wording --- versions/development/SPEC.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/versions/development/SPEC.md b/versions/development/SPEC.md index c040cfeb..2ceb52c8 100644 --- a/versions/development/SPEC.md +++ b/versions/development/SPEC.md @@ -2619,9 +2619,11 @@ form `workflow_name.input_name`. `{ inputs: my_task.my_task_input=... }` * All **required** inputs which bubble up like this must ultimately be provided in the inputs set before the top-level workflow can be run. - * An engine must allow these optional bubbled-up inputs to be filled in by end - users. However, it is up to the engine how many layers of nesting it chooses - to expose in the UI. +* An engine must allow **optional** bubbled-up inputs to be filled in by end + users. These inputs are not required to be filled. +* Showing all optional inputs which are deeply nested in the workflow can lead + to a very large number of inputs. Therefore, it is up to the engine how many + layers of nesting it chooses to make visible in the UI. Any declaration that appears outside the `input` section is considered an intermediate value and **not** a workflow input. Any declaration can always be moved inside the `input` block to make it overridable.