From e890e79b1b97718ca6f35800a16fc3c67117aec3 Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Wed, 16 Sep 2020 11:05:45 +0300 Subject: [PATCH 01/11] Draft version of the Loop-5 operation specification --- docs/ops/infrastructure/Loop_5.md | 395 ++++++++++++++++++++++++++++++ 1 file changed, 395 insertions(+) create mode 100644 docs/ops/infrastructure/Loop_5.md diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md new file mode 100644 index 00000000000000..a40d8d8bd22296 --- /dev/null +++ b/docs/ops/infrastructure/Loop_5.md @@ -0,0 +1,395 @@ +## Loop {#openvino_docs_ops_infrastructure_Loop_5} + +**Versioned name**: *Loop-5* + +**Category**: Infrastructure + +**Short description**: *Loop* operation performs recurrent execution of the network, which is described in the `body`, iterating through the data. +The operation has similar semantic to the ONNX* Loop [operation](https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Loop-13). + +**Detailed description** + +The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count" and "termination condition". + +1. Trip count input is an integer scalar input specifying maximum number of iterations. //Default value is "-1" meaning infinite number of iterations. +2. Loop termination condition input is a boolean scalar input specifying whether to run the current loop iteration or not. +Note, that the body of the Loop must yield the termination condition value whether the input is provided or not. + +Both inputs are optional. +//Default value is a "True" constant meaning need to perform first iteration. + +There are several combinations of these two inputs which are described in the following code snippets (empty string `""` means no input provided): + +``` + input ("", ""): + for (int i = 0; ; ++i) + { + cond = ...; // the "cond" value must be yield by the body, but is ignored in the loop condition check + } + + input ("", cond) // while loop + bool cond = ...; + for (int i = 0; cond; ++i) + { + cond = ...; + } + + input ("", 1) // do-while loop + bool cond = true; + for (int i = 0; cond; ++i) + { + cond = ...; + } + + input (trip_count, "") // for loop + int trip_count = ...; + for (int i = 0; i < trip_count; ++i) + { + cond = ...; // the "cond" value must be yield by the body, but is ignored in the loop condition check + } + + input (trip_count, cond) + int trip_count = ...; + bool cond = ...; + for (int i = 0; i < trip_count && cond; ++i) + { + cond = ...; + } +``` + +The body graph has two inputs: +1. The "current iteration" number. +2. The "termination condition". This value is provided from the corresponding input of the Loop operation for the first iteration and calculated in the body graph for the consequent iterations. + +Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify termination condition(s). +Loop operation description in the IR also has several special sections: `body`, `port_map` and `back_edges` similar to the ones from the TensorIterator operation but having some important features described below. + +1. The body operation getting an input from the main graph should have an entry in the `port_map` section of the Loop operation. These edges connect input ports of the Loop with the body `Parameter`s. +1. The body operation producing tensor to be used in the subsequent iterations (like in RNN models) should have a back edge described in the `back_edges` section of the operation. The back edge connects the respective body `Parameter` and `Result` operations. For such a case the Loop operation node provides input for the first iteration, while corresponding Loop operation output produces the tensor computed during the last iteration. +1. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output. The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). +1. There is one body `Parameter` operation not connected through the `port_map`. This is a "current iteration" input. The Loop operation is responsible for providing the appropriate value for each iteration. +1. Connection of nodes inside the Loop with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. + +**Loop attributes**: + +* **Body**: + + `body` is a network that will be recurrently executed. The network is described operation by operation as a typical IR network. + + * **Body attributes**: + + No attributes available. + +* **Port map**: + + *port_map* is a set of rules to map input or output data tensors of `Loop` operation onto `body` data tensors. The `port_map` entries can be` input` and `output`. Each entry describes a corresponding mapping rule. + + * **Port map attributes**: + + * *external_port_id* + * **Description**: *external_port_id* is a port ID of the `Loop` operation. + * **Range of values**: indexes of the *Loop* outputs + * **Type**: `int` + * **Default value**: None + * **Required**: *yes* + + * *internal_operation_id* + + * **Description**: *internal_operation_id* is a `Parameter` or `Result` operation ID inside the `body` network to map to. + * **Range of values**: IDs of the `Parameter` operations inside in the *Loop* operation + * **Type**: `int` + * **Default value**: None + * **Required**: *yes* + + * *axis* + + * **Description**: *axis* is an axis to concatenate the body `Result` output across all iterations. Can be specified for `output` edges only. + * **Range of values**: an integer + * **Type**: `int` + * **Default value**: None + * **Required**: *no* + +* **Back edges**: + + *back_edges* is a set of rules to transfer tensor values from `body` outputs at one iteration to `body` parameters at the next iteration. Back edge connects some `Result` operation in the `body` to `Parameter` operation in the same `body`. + + * **Back edge attributes**: + + * *from-operation* + + * **Description**: *from-operation* is a `Result` operation ID inside the `body` network. + * **Range of values**: IDs of the `Result` operations inside the *Loop* + * **Type**: `int` + * **Default value**: None + * **Required**: *yes* + + * *to-operation* + + * **Description**: *to-operation* is a `Parameter` operation ID inside the `body` network to end mapping. + * **Range of values**: IDs of the `Parameter` operations inside the *Loop* + * **Type**: `int` + * **Default value**: None + * **Required**: *yes* + +**Loop Inputs** + +* **Trip count**: A scalar tensor of `int64` type specifying maximum number of iterations. *Optional*. + +* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the first iteration or not. *Optional*. + +* **Multiple other inputs**: tensors of different types and shapes. *Optional*. + +**Loop Outputs** + +* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape. + + +**Body Inputs** + +* **Current iteration**: A scalar tensor of `int64` type specifying the current iteration number. *Optional*. + +* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the current iteration or not. *Optional*. + +* **Multiple other inputs**: tensors of different types and shapes. *Optional*. + +**Body Outputs** + +* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the next iteration or not. + +* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape. + + +**Examples** + +*Example 1: a typical Loop structure* +```xml + + ... + ... + + + + ... + + ... + + + + ... + + + ... + ... + + +``` + +*Example 2: a full Loop operation* + +```xml + + + + 1 + 25 + 512 + + + 1 + 256 + + + 1 + 256 + + + + + 1 + 25 + 256 + + + + + + + + + + + + + + + + + + 1 + 1 + 512 + + + + + + + + 2 + + + + + + + 1 + 1 + 512 + + + 2 + + + + + 1 + 512 + + + + + + + 1 + 256 + + + + + + + 1 + 256 + + + + + + + + 1024 + 768 + + + + + + + + 1024 + + + + + + + + 1 + 512 + + + 1 + 256 + + + 1 + 256 + + + 1024 + 768 + + + 1024 + + + + + 1 + 256 + + + 1 + 256 + + + + + + + 1 + 256 + + + + + + + 1 + 256 + + + + + + + + 3 + + + + + + + 1 + 256 + + + 3 + + + + + 1 + 1 + 256 + + + + + + + 1 + 1 + 256 + + + + + + + + + + + + + + + + + + + + +``` \ No newline at end of file From 174d9f23e6fe977fcaedb2ef85e36e47e326430e Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Wed, 16 Sep 2020 18:34:09 +0300 Subject: [PATCH 02/11] Draft version of the Loop-5 operation specification --- docs/ops/infrastructure/Loop_5.md | 249 +++--------------------------- 1 file changed, 23 insertions(+), 226 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index a40d8d8bd22296..1ed9af8fa94feb 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -11,14 +11,11 @@ The operation has similar semantic to the ONNX* Loop [operation](https://github. The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count" and "termination condition". -1. Trip count input is an integer scalar input specifying maximum number of iterations. //Default value is "-1" meaning infinite number of iterations. -2. Loop termination condition input is a boolean scalar input specifying whether to run the current loop iteration or not. +1. Trip count input is an integer scalar input specifying maximum number of iterations. When this input is not connected perform infinite number of iterations. +2. Loop termination condition input is a boolean scalar input specifying whether to run the current loop iteration or not. When this input is not connected perform first iteration. Note, that the body of the Loop must yield the termination condition value whether the input is provided or not. -Both inputs are optional. -//Default value is a "True" constant meaning need to perform first iteration. - -There are several combinations of these two inputs which are described in the following code snippets (empty string `""` means no input provided): +There are several combinations of these two inputs which are described in the following code snippet (empty string `""` means no input provided): ``` input ("", ""): @@ -57,18 +54,19 @@ There are several combinations of these two inputs which are described in the fo } ``` -The body graph has two inputs: -1. The "current iteration" number. -2. The "termination condition". This value is provided from the corresponding input of the Loop operation for the first iteration and calculated in the body graph for the consequent iterations. +The body graph has at least two inputs: +1. The "current iteration" number which is an integer scalar number. +2. The "termination condition" which is a boolean scalar value. This value is provided from the corresponding input of the Loop operation for the first iteration and calculated in the body graph for the consequent iterations. Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify termination condition(s). Loop operation description in the IR also has several special sections: `body`, `port_map` and `back_edges` similar to the ones from the TensorIterator operation but having some important features described below. 1. The body operation getting an input from the main graph should have an entry in the `port_map` section of the Loop operation. These edges connect input ports of the Loop with the body `Parameter`s. 1. The body operation producing tensor to be used in the subsequent iterations (like in RNN models) should have a back edge described in the `back_edges` section of the operation. The back edge connects the respective body `Parameter` and `Result` operations. For such a case the Loop operation node provides input for the first iteration, while corresponding Loop operation output produces the tensor computed during the last iteration. -1. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output. The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). +1. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output (this is a "scan output" according to the ONNX* Loop operation [specification](https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Loop-13)). The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). 1. There is one body `Parameter` operation not connected through the `port_map`. This is a "current iteration" input. The Loop operation is responsible for providing the appropriate value for each iteration. -1. Connection of nodes inside the Loop with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. +1. The body `Parameter` operation corresponding to termination condition must have attribute `"condition=True"` in the corresponding `port_map` entry. If termination condition input is not connected then no entries should have this attribute set. +1. Connection of nodes inside the Loop body with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. **Loop attributes**: @@ -82,7 +80,7 @@ Loop operation description in the IR also has several special sections: `body`, * **Port map**: - *port_map* is a set of rules to map input or output data tensors of `Loop` operation onto `body` data tensors. The `port_map` entries can be` input` and `output`. Each entry describes a corresponding mapping rule. + *port_map* is a set of rules to map input or output data tensors of `Loop` operation onto `body` data tensors. The `port_map` entries can be` input` and `output`. Each entry describes a corresponding mapping rule. * **Port map attributes**: @@ -109,6 +107,14 @@ Loop operation description in the IR also has several special sections: `body`, * **Default value**: None * **Required**: *no* + * *condition* + + * **Description**: *condition* is a boolean flag to mark input as a termination condition input. Can be specified as `True` for only one `input` edge. + * **Range of values**: True or False + * **Type**: `boolean` + * **Default value**: False + * **Required**: *no* + * **Back edges**: *back_edges* is a set of rules to transfer tensor values from `body` outputs at one iteration to `body` parameters at the next iteration. Back edge connects some `Result` operation in the `body` to `Parameter` operation in the same `body`. @@ -146,12 +152,13 @@ Loop operation description in the IR also has several special sections: `body`, **Body Inputs** -* **Current iteration**: A scalar tensor of `int64` type specifying the current iteration number. *Optional*. +* **Current iteration**: A scalar tensor of `int64` type specifying the current iteration number. *Required*. -* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the current iteration or not. *Optional*. +* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the current iteration or not. *Required*. * **Multiple other inputs**: tensors of different types and shapes. *Optional*. + **Body Outputs** * **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the next iteration or not. @@ -167,10 +174,10 @@ Loop operation description in the IR also has several special sections: `body`, ... ... - + ... - + ... @@ -183,213 +190,3 @@ Loop operation description in the IR also has several special sections: `body`, ``` - -*Example 2: a full Loop operation* - -```xml - - - - 1 - 25 - 512 - - - 1 - 256 - - - 1 - 256 - - - - - 1 - 25 - 256 - - - - - - - - - - - - - - - - - - 1 - 1 - 512 - - - - - - - - 2 - - - - - - - 1 - 1 - 512 - - - 2 - - - - - 1 - 512 - - - - - - - 1 - 256 - - - - - - - 1 - 256 - - - - - - - - 1024 - 768 - - - - - - - - 1024 - - - - - - - - 1 - 512 - - - 1 - 256 - - - 1 - 256 - - - 1024 - 768 - - - 1024 - - - - - 1 - 256 - - - 1 - 256 - - - - - - - 1 - 256 - - - - - - - 1 - 256 - - - - - - - - 3 - - - - - - - 1 - 256 - - - 3 - - - - - 1 - 1 - 256 - - - - - - - 1 - 1 - 256 - - - - - - - - - - - - - - - - - - - - -``` \ No newline at end of file From 3402577de7a707cfd1ebe2e115dbbea28b34a0cd Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Wed, 16 Sep 2020 18:39:51 +0300 Subject: [PATCH 03/11] Example modification --- docs/ops/infrastructure/Loop_5.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index 1ed9af8fa94feb..266c1caf91b0bd 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -174,8 +174,8 @@ Loop operation description in the IR also has several special sections: `body`, ... ... - - + + ... ... From c6d6880fe0b94365657e06a5454e8cde4298683d Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Thu, 17 Sep 2020 11:17:14 +0300 Subject: [PATCH 04/11] Update for the Loop-5 specification --- docs/ops/infrastructure/Loop_5.md | 47 +++++++++++-------------------- 1 file changed, 17 insertions(+), 30 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index 266c1caf91b0bd..497c463dea4423 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -9,43 +9,38 @@ The operation has similar semantic to the ONNX* Loop [operation](https://github. **Detailed description** -The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count" and "termination condition". +The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count" and "termination condition" and inputs of Loop body called "current iteration" and "termination condition". -1. Trip count input is an integer scalar input specifying maximum number of iterations. When this input is not connected perform infinite number of iterations. -2. Loop termination condition input is a boolean scalar input specifying whether to run the current loop iteration or not. When this input is not connected perform first iteration. -Note, that the body of the Loop must yield the termination condition value whether the input is provided or not. +The Loop inputs have the following meaning: +1. Trip count input is an integer scalar input specifying maximum number of iterations. To simulate infinite loop provide Constant `-1` as input. +2. Loop termination condition input is a boolean scalar input specifying whether to run the current loop iteration or not. Note, that the body of the Loop must yield the termination condition value. -There are several combinations of these two inputs which are described in the following code snippet (empty string `""` means no input provided): +There are several combinations of these two inputs `(trip_count, condition)` which are described in the following code snippet: ``` - input ("", ""): - for (int i = 0; ; ++i) - { - cond = ...; // the "cond" value must be yield by the body, but is ignored in the loop condition check - } - - input ("", cond) // while loop + input (-1, cond) // while loop bool cond = ...; for (int i = 0; cond; ++i) { cond = ...; } - input ("", 1) // do-while loop + input (-1, true) // do-while loop bool cond = true; for (int i = 0; cond; ++i) { cond = ...; } - input (trip_count, "") // for loop + input (trip_count, true) // for loop int trip_count = ...; + bool cond = true; for (int i = 0; i < trip_count; ++i) { - cond = ...; // the "cond" value must be yield by the body, but is ignored in the loop condition check + cond = true; // sub-graph calculating condition must always return "true"! } - input (trip_count, cond) + input (trip_count, cond) // for with condition int trip_count = ...; bool cond = ...; for (int i = 0; i < trip_count && cond; ++i) @@ -54,8 +49,8 @@ There are several combinations of these two inputs which are described in the fo } ``` -The body graph has at least two inputs: -1. The "current iteration" number which is an integer scalar number. +The body graph inputs have the following meaning: +1. The "current iteration" number which is an integer scalar number. The iteration number starts from 0 and incremented by one for each iteration. 2. The "termination condition" which is a boolean scalar value. This value is provided from the corresponding input of the Loop operation for the first iteration and calculated in the body graph for the consequent iterations. Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify termination condition(s). @@ -65,7 +60,6 @@ Loop operation description in the IR also has several special sections: `body`, 1. The body operation producing tensor to be used in the subsequent iterations (like in RNN models) should have a back edge described in the `back_edges` section of the operation. The back edge connects the respective body `Parameter` and `Result` operations. For such a case the Loop operation node provides input for the first iteration, while corresponding Loop operation output produces the tensor computed during the last iteration. 1. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output (this is a "scan output" according to the ONNX* Loop operation [specification](https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Loop-13)). The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). 1. There is one body `Parameter` operation not connected through the `port_map`. This is a "current iteration" input. The Loop operation is responsible for providing the appropriate value for each iteration. -1. The body `Parameter` operation corresponding to termination condition must have attribute `"condition=True"` in the corresponding `port_map` entry. If termination condition input is not connected then no entries should have this attribute set. 1. Connection of nodes inside the Loop body with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. **Loop attributes**: @@ -107,14 +101,6 @@ Loop operation description in the IR also has several special sections: `body`, * **Default value**: None * **Required**: *no* - * *condition* - - * **Description**: *condition* is a boolean flag to mark input as a termination condition input. Can be specified as `True` for only one `input` edge. - * **Range of values**: True or False - * **Type**: `boolean` - * **Default value**: False - * **Required**: *no* - * **Back edges**: *back_edges* is a set of rules to transfer tensor values from `body` outputs at one iteration to `body` parameters at the next iteration. Back edge connects some `Result` operation in the `body` to `Parameter` operation in the same `body`. @@ -139,9 +125,9 @@ Loop operation description in the IR also has several special sections: `body`, **Loop Inputs** -* **Trip count**: A scalar tensor of `int64` type specifying maximum number of iterations. *Optional*. +* **Trip count**: A scalar tensor of `int64` type specifying maximum number of iterations. *Required*. -* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the first iteration or not. *Optional*. +* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the first iteration or not. *Required*. * **Multiple other inputs**: tensors of different types and shapes. *Optional*. @@ -175,9 +161,10 @@ Loop operation description in the IR also has several special sections: `body`, ... - + ... + ... From c5bcca9659f47f77e086e26259e019476782f199 Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Thu, 17 Sep 2020 17:00:05 +0300 Subject: [PATCH 05/11] Specification update --- docs/ops/infrastructure/Loop_5.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index 497c463dea4423..773344a48d594c 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -9,15 +9,22 @@ The operation has similar semantic to the ONNX* Loop [operation](https://github. **Detailed description** -The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count" and "termination condition" and inputs of Loop body called "current iteration" and "termination condition". +The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count", "termination condition" and inputs of Loop body called "current iteration". -The Loop inputs have the following meaning: -1. Trip count input is an integer scalar input specifying maximum number of iterations. To simulate infinite loop provide Constant `-1` as input. -2. Loop termination condition input is a boolean scalar input specifying whether to run the current loop iteration or not. Note, that the body of the Loop must yield the termination condition value. +These Loop operation inputs have the following meaning: +1. Trip count is an integer scalar input specifying maximum number of iterations. To simulate infinite loop Constant `-1` can be provided as input. +2. Loop termination condition input is a boolean scalar input specifying whether to run the first loop iteration or not. Note, that the body of the Loop must yield the termination condition value for the consecutive iterations. There are several combinations of these two inputs `(trip_count, condition)` which are described in the following code snippet: ``` + input (-1, true) // infinite loop + bool cond = true; + for (int i = 0; cond; ++i) + { + cond = true; // sub-graph calculating condition must always return "true"! + } + input (-1, cond) // while loop bool cond = ...; for (int i = 0; cond; ++i) @@ -49,9 +56,8 @@ There are several combinations of these two inputs `(trip_count, condition)` whi } ``` -The body graph inputs have the following meaning: -1. The "current iteration" number which is an integer scalar number. The iteration number starts from 0 and incremented by one for each iteration. -2. The "termination condition" which is a boolean scalar value. This value is provided from the corresponding input of the Loop operation for the first iteration and calculated in the body graph for the consequent iterations. +1. The body graph first input called "current iteration" is an integer scalar number specifying current iteration number. The iteration number starts from 0 and incremented by one for each iteration. +1. The body graph first output called "termination condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify termination condition(s). Loop operation description in the IR also has several special sections: `body`, `port_map` and `back_edges` similar to the ones from the TensorIterator operation but having some important features described below. @@ -95,7 +101,7 @@ Loop operation description in the IR also has several special sections: `body`, * *axis* - * **Description**: *axis* is an axis to concatenate the body `Result` output across all iterations. Can be specified for `output` edges only. + * **Description**: *axis* is an axis to concatenate the body `Result` output across all iterations. Can be specified for `output` entry only. * **Range of values**: an integer * **Type**: `int` * **Default value**: None @@ -140,8 +146,6 @@ Loop operation description in the IR also has several special sections: `body`, * **Current iteration**: A scalar tensor of `int64` type specifying the current iteration number. *Required*. -* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the current iteration or not. *Required*. - * **Multiple other inputs**: tensors of different types and shapes. *Optional*. From 4b91b50971af4e852fdc282e567b4847583c9595 Mon Sep 17 00:00:00 2001 From: Ivan Tikhonov Date: Tue, 29 Sep 2020 16:38:56 +0300 Subject: [PATCH 06/11] Resolve review remarks --- docs/ops/infrastructure/Loop_5.md | 48 +++++++++++++++---------------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index 773344a48d594c..6d54f41ad2315f 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -57,16 +57,16 @@ There are several combinations of these two inputs `(trip_count, condition)` whi ``` 1. The body graph first input called "current iteration" is an integer scalar number specifying current iteration number. The iteration number starts from 0 and incremented by one for each iteration. -1. The body graph first output called "termination condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. +2. The body graph first output called "termination condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify termination condition(s). Loop operation description in the IR also has several special sections: `body`, `port_map` and `back_edges` similar to the ones from the TensorIterator operation but having some important features described below. -1. The body operation getting an input from the main graph should have an entry in the `port_map` section of the Loop operation. These edges connect input ports of the Loop with the body `Parameter`s. -1. The body operation producing tensor to be used in the subsequent iterations (like in RNN models) should have a back edge described in the `back_edges` section of the operation. The back edge connects the respective body `Parameter` and `Result` operations. For such a case the Loop operation node provides input for the first iteration, while corresponding Loop operation output produces the tensor computed during the last iteration. -1. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output (this is a "scan output" according to the ONNX* Loop operation [specification](https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Loop-13)). The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). -1. There is one body `Parameter` operation not connected through the `port_map`. This is a "current iteration" input. The Loop operation is responsible for providing the appropriate value for each iteration. -1. Connection of nodes inside the Loop body with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. +3. The body operation getting an input from the main graph should have an entry in the `port_map` section of the Loop operation. These edges connect input ports of the Loop with the body `Parameter`s. +4. The body operation producing tensor to be used in the subsequent iterations (like in RNN models) should have a back edge described in the `back_edges` section of the operation. The back edge connects the respective body `Parameter` and `Result` operations. For such a case the Loop operation node provides input for the first iteration, while corresponding Loop operation output produces the tensor computed during the last iteration. +5. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output (this is a "scan output" according to the ONNX* Loop operation [specification](https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Loop-13)). The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). +6. There is one body `Parameter` operation not connected through the `port_map`. This is a "current iteration" input. The Loop operation is responsible for providing the appropriate value for each iteration. +7. Connection of nodes inside the Loop body with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. **Loop attributes**: @@ -86,14 +86,14 @@ Loop operation description in the IR also has several special sections: `body`, * *external_port_id* * **Description**: *external_port_id* is a port ID of the `Loop` operation. - * **Range of values**: indexes of the *Loop* outputs + * **Range of values**: IDs of the *Loop* outputs * **Type**: `int` * **Default value**: None * **Required**: *yes* - * *internal_operation_id* + * *internal_layer_id* - * **Description**: *internal_operation_id* is a `Parameter` or `Result` operation ID inside the `body` network to map to. + * **Description**: *internal_layer_id* is a `Parameter` or `Result` operation ID inside the `body` network to map to. * **Range of values**: IDs of the `Parameter` operations inside in the *Loop* operation * **Type**: `int` * **Default value**: None @@ -102,7 +102,7 @@ Loop operation description in the IR also has several special sections: `body`, * *axis* * **Description**: *axis* is an axis to concatenate the body `Result` output across all iterations. Can be specified for `output` entry only. - * **Range of values**: an integer + * **Range of values**: an integer. Negative value means counting dimension from the end. * **Type**: `int` * **Default value**: None * **Required**: *no* @@ -113,17 +113,17 @@ Loop operation description in the IR also has several special sections: `body`, * **Back edge attributes**: - * *from-operation* + * *from-layer* - * **Description**: *from-operation* is a `Result` operation ID inside the `body` network. + * **Description**: *from-layer* is a `Result` operation ID inside the `body` network. * **Range of values**: IDs of the `Result` operations inside the *Loop* * **Type**: `int` * **Default value**: None * **Required**: *yes* - * *to-operation* + * *to-layer* - * **Description**: *to-operation* is a `Parameter` operation ID inside the `body` network to end mapping. + * **Description**: *to-layer* is a `Parameter` operation ID inside the `body` network to end mapping. * **Range of values**: IDs of the `Parameter` operations inside the *Loop* * **Type**: `int` * **Default value**: None @@ -133,7 +133,7 @@ Loop operation description in the IR also has several special sections: `body`, * **Trip count**: A scalar tensor of `int64` type specifying maximum number of iterations. *Required*. -* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the first iteration or not. *Required*. +* **Condition**: A scalar tensor of `boolean` type specifying whether to execute the first iteration or not. `True` means to execute the 1st iteration. *Required*. * **Multiple other inputs**: tensors of different types and shapes. *Optional*. @@ -151,7 +151,7 @@ Loop operation description in the IR also has several special sections: `body`, **Body Outputs** -* **Termination condition**: A scalar tensor of `boolean` type specifying whether to execute the next iteration or not. +* **Condition**: A scalar tensor of `boolean` type specifying whether to execute the next iteration or not. `True` means to continue execution. * **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape. @@ -160,24 +160,24 @@ Loop operation description in the IR also has several special sections: `body`, *Example 1: a typical Loop structure* ```xml - + ... ... - - + + ... - - + + ... - + ... - ... + ... ... - + ``` From cf5c3f8d4190d630445f01358a49bdfd7facb330 Mon Sep 17 00:00:00 2001 From: Ivan Tikhonov Date: Tue, 29 Sep 2020 16:48:45 +0300 Subject: [PATCH 07/11] Renaming from termination condition to condition --- docs/ops/infrastructure/Loop_5.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index 6d54f41ad2315f..8a4923811b49b0 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -9,11 +9,11 @@ The operation has similar semantic to the ONNX* Loop [operation](https://github. **Detailed description** -The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count", "termination condition" and inputs of Loop body called "current iteration". +The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count", "condition" and inputs of Loop body called "current iteration". These Loop operation inputs have the following meaning: 1. Trip count is an integer scalar input specifying maximum number of iterations. To simulate infinite loop Constant `-1` can be provided as input. -2. Loop termination condition input is a boolean scalar input specifying whether to run the first loop iteration or not. Note, that the body of the Loop must yield the termination condition value for the consecutive iterations. +2. Loop condition input is a boolean scalar input specifying whether to run the first loop iteration or not. Note, that the body of the Loop must yield the condition value for the consecutive iterations. There are several combinations of these two inputs `(trip_count, condition)` which are described in the following code snippet: @@ -57,9 +57,9 @@ There are several combinations of these two inputs `(trip_count, condition)` whi ``` 1. The body graph first input called "current iteration" is an integer scalar number specifying current iteration number. The iteration number starts from 0 and incremented by one for each iteration. -2. The body graph first output called "termination condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. +2. The body graph first output called "condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. -Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify termination condition(s). +Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify condition(s). Loop operation description in the IR also has several special sections: `body`, `port_map` and `back_edges` similar to the ones from the TensorIterator operation but having some important features described below. 3. The body operation getting an input from the main graph should have an entry in the `port_map` section of the Loop operation. These edges connect input ports of the Loop with the body `Parameter`s. From 9f890c02e9105774686adcce809c91f66f0981c8 Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Thu, 8 Oct 2020 12:28:57 +0300 Subject: [PATCH 08/11] Updated Loop specification --- docs/ops/infrastructure/Loop_5.md | 32 +++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index 8a4923811b49b0..4c2b702f518fcb 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -9,11 +9,11 @@ The operation has similar semantic to the ONNX* Loop [operation](https://github. **Detailed description** -The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count", "condition" and inputs of Loop body called "current iteration". +The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count", "condition" and input of the Loop body called "current iteration". These Loop operation inputs have the following meaning: -1. Trip count is an integer scalar input specifying maximum number of iterations. To simulate infinite loop Constant `-1` can be provided as input. -2. Loop condition input is a boolean scalar input specifying whether to run the first loop iteration or not. Note, that the body of the Loop must yield the condition value for the consecutive iterations. +1. Trip count is an integer scalar or 1D tensor with 1 element input specifying maximum number of iterations. To simulate infinite loop Constant `-1` can be provided as input. +2. Loop condition input is a boolean scalar or 1D tensor with 1 element input specifying whether to run the first loop iteration or not. Note, that the body of the Loop must yield the condition value for the consecutive iterations. There are several combinations of these two inputs `(trip_count, condition)` which are described in the following code snippet: @@ -56,17 +56,17 @@ There are several combinations of these two inputs `(trip_count, condition)` whi } ``` -1. The body graph first input called "current iteration" is an integer scalar number specifying current iteration number. The iteration number starts from 0 and incremented by one for each iteration. -2. The body graph first output called "condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. +1. One of the body graph inputs called "current iteration" is an integer scalar or 1D integer tensor with 1 number specifying current iteration number. The iteration number starts from 0 and incremented by one for each iteration. This input is optional and may not exist if the iteration number value is not used in the body. +2. One of the body graph outputs is called "condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify condition(s). Loop operation description in the IR also has several special sections: `body`, `port_map` and `back_edges` similar to the ones from the TensorIterator operation but having some important features described below. -3. The body operation getting an input from the main graph should have an entry in the `port_map` section of the Loop operation. These edges connect input ports of the Loop with the body `Parameter`s. -4. The body operation producing tensor to be used in the subsequent iterations (like in RNN models) should have a back edge described in the `back_edges` section of the operation. The back edge connects the respective body `Parameter` and `Result` operations. For such a case the Loop operation node provides input for the first iteration, while corresponding Loop operation output produces the tensor computed during the last iteration. -5. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output (this is a "scan output" according to the ONNX* Loop operation [specification](https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Loop-13)). The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). -6. There is one body `Parameter` operation not connected through the `port_map`. This is a "current iteration" input. The Loop operation is responsible for providing the appropriate value for each iteration. -7. Connection of nodes inside the Loop body with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. +1. The body operation getting an input from the main graph should have an entry in the `port_map` section of the Loop operation. These edges connect input ports of the Loop with the body `Parameter`s. +2. The body operation producing tensor to be used in the subsequent iterations (like in RNN models) should have a back edge described in the `back_edges` section of the operation. The back edge connects the respective body `Parameter` and `Result` operations. For such a case the Loop operation node provides input for the first iteration, while corresponding Loop operation output produces the tensor computed during the last iteration. +3. Output tensors produced by a particular body operation across all iterations can be concatenated and returned as a Loop operation output (this is a "scan output" according to the ONNX* Loop operation [specification](https://github.com/onnx/onnx/blob/master/docs/Changelog.md#Loop-13)). The corresponding `output` entry in the `port_map` should have `axis` attribute specifying the axis to concatenate. Therefore, outputs from operations corresponding to `output` entries in the `port_map` without `axis` attribute are returned "as is" (without concatenation). +4. There is one body `Parameter` operation not connected through the `port_map`. This is a "current iteration" input. The Loop operation is responsible for providing the appropriate value for each iteration. +5. Connection of nodes inside the Loop body with the main graph should be done through `Parameter` and `Result` body operations. No other ways to connect graphs are allowed. **Loop attributes**: @@ -144,16 +144,14 @@ Loop operation description in the IR also has several special sections: `body`, **Body Inputs** -* **Current iteration**: A scalar tensor of `int64` type specifying the current iteration number. *Required*. - -* **Multiple other inputs**: tensors of different types and shapes. *Optional*. +* **Multiple inputs**: tensors of different types and shapes. One of the inputs corresponds to current iteration number input (this input is optional). This input is marked in the port_map with attribute `purpose = "iteration_number"`. *Optional*. **Body Outputs** * **Condition**: A scalar tensor of `boolean` type specifying whether to execute the next iteration or not. `True` means to continue execution. -* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape. +* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape. One of the outputs corresponds to the output with execution condition. This output is marked in the port_map with attribute `purpose = "execution_condition"`. **Examples** @@ -166,13 +164,15 @@ Loop operation description in the IR also has several special sections: `body`, + ... - + + ... - + ... From 727832904099b29ae7a1f14b7a4acd10785a064e Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Thu, 8 Oct 2020 12:49:54 +0300 Subject: [PATCH 09/11] Updated necessary documents for Loop specification --- docs/doxygen/ie_docs.xml | 1 + docs/ops/opset5.md | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/doxygen/ie_docs.xml b/docs/doxygen/ie_docs.xml index 97d842568a31d7..18544111d4a06a 100644 --- a/docs/doxygen/ie_docs.xml +++ b/docs/doxygen/ie_docs.xml @@ -170,6 +170,7 @@ + diff --git a/docs/ops/opset5.md b/docs/ops/opset5.md index c2083aa39d0fca..ec1219993581a1 100644 --- a/docs/ops/opset5.md +++ b/docs/ops/opset5.md @@ -73,6 +73,7 @@ declared in `namespace opset5`. * [LogicalNot](logical/LogicalNot_1.md) * [LogicalOr](logical/LogicalOr_1.md) * [LogicalXor](logical/LogicalXor_1.md) +* [Loop](infrastructure/Loop_5.md) * [LRN](normalization/LRN_1.md) * [LSTMCell](sequence/LSTMCell_1.md) * [LSTMSequence](sequence/LSTMSequence_1.md) From 8877db1b0db0ef3359e7f867c6a8348d90ecfba4 Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Thu, 8 Oct 2020 18:13:45 +0300 Subject: [PATCH 10/11] Updated Loop specification --- docs/ops/infrastructure/Loop_5.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index 4c2b702f518fcb..d0483f867e8509 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -9,13 +9,13 @@ The operation has similar semantic to the ONNX* Loop [operation](https://github. **Detailed description** -The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count", "condition" and input of the Loop body called "current iteration". +The body of the Loop can be executed 0 or more times depending on the values passed to the Loop operation inputs called "trip count", "execution condition" and input of the Loop body called "current iteration". These Loop operation inputs have the following meaning: 1. Trip count is an integer scalar or 1D tensor with 1 element input specifying maximum number of iterations. To simulate infinite loop Constant `-1` can be provided as input. -2. Loop condition input is a boolean scalar or 1D tensor with 1 element input specifying whether to run the first loop iteration or not. Note, that the body of the Loop must yield the condition value for the consecutive iterations. +2. Loop execution condition input is a boolean scalar or 1D tensor with 1 element input specifying whether to run the first loop iteration or not. Note, that the body of the Loop must yield the condition value for the consecutive iterations. -There are several combinations of these two inputs `(trip_count, condition)` which are described in the following code snippet: +There are several combinations of these two inputs `(trip_count, execution condition)` which are described in the following code snippet: ``` input (-1, true) // infinite loop @@ -57,7 +57,7 @@ There are several combinations of these two inputs `(trip_count, condition)` whi ``` 1. One of the body graph inputs called "current iteration" is an integer scalar or 1D integer tensor with 1 number specifying current iteration number. The iteration number starts from 0 and incremented by one for each iteration. This input is optional and may not exist if the iteration number value is not used in the body. -2. One of the body graph outputs is called "condition" is a boolean scalar value. This value is used to decide whenever to perform the next iteration or not. +2. One of the body graph outputs is called "condition" is a boolean scalar or 1D tensor with 1 element. This value is used to decide whenever to perform the next iteration or not. Loop operation description in the IR has regular sections: `input` and `output`. They connect Loop body to the outer graph and specify condition(s). Loop operation description in the IR also has several special sections: `body`, `port_map` and `back_edges` similar to the ones from the TensorIterator operation but having some important features described below. @@ -131,9 +131,9 @@ Loop operation description in the IR also has several special sections: `body`, **Loop Inputs** -* **Trip count**: A scalar tensor of `int64` type specifying maximum number of iterations. *Required*. +* **Trip count**: A scalar or 1D tensor with 1 element of `int64` or `int32` type specifying maximum number of iterations. *Required*. -* **Condition**: A scalar tensor of `boolean` type specifying whether to execute the first iteration or not. `True` means to execute the 1st iteration. *Required*. +* **ExecutionCondition**: A scalar or 1D tensor with 1 element of `boolean` type specifying whether to execute the first iteration or not. `True` value means to execute the 1st iteration. *Required*. * **Multiple other inputs**: tensors of different types and shapes. *Optional*. @@ -144,13 +144,11 @@ Loop operation description in the IR also has several special sections: `body`, **Body Inputs** -* **Multiple inputs**: tensors of different types and shapes. One of the inputs corresponds to current iteration number input (this input is optional). This input is marked in the port_map with attribute `purpose = "iteration_number"`. *Optional*. +* **Multiple inputs**: tensors of different types and shapes. One of the inputs corresponds to current iteration number input (this input is optional). This input is marked in the port_map with attribute `purpose = "current_iteration"`. *Optional*. **Body Outputs** -* **Condition**: A scalar tensor of `boolean` type specifying whether to execute the next iteration or not. `True` means to continue execution. - * **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape. One of the outputs corresponds to the output with execution condition. This output is marked in the port_map with attribute `purpose = "execution_condition"`. @@ -164,7 +162,7 @@ Loop operation description in the IR also has several special sections: `body`, - + ... From 0293f695715b863dc06685e727815336517e2151 Mon Sep 17 00:00:00 2001 From: Evgeny Lazarev Date: Mon, 12 Oct 2020 14:02:28 +0300 Subject: [PATCH 11/11] Updated Loop-5 specification --- docs/ops/infrastructure/Loop_5.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/ops/infrastructure/Loop_5.md b/docs/ops/infrastructure/Loop_5.md index d0483f867e8509..3b6ad094e44eec 100644 --- a/docs/ops/infrastructure/Loop_5.md +++ b/docs/ops/infrastructure/Loop_5.md @@ -144,12 +144,12 @@ Loop operation description in the IR also has several special sections: `body`, **Body Inputs** -* **Multiple inputs**: tensors of different types and shapes. One of the inputs corresponds to current iteration number input (this input is optional). This input is marked in the port_map with attribute `purpose = "current_iteration"`. *Optional*. +* **Multiple inputs**: tensors of different types and shapes except the one corresponding to the current iteration number. This input is marked in the port_map with attribute `purpose = "current_iteration"` and produces a scalar or 1D tensor with 1 element of `int64` or `int32` type. *Optional*. **Body Outputs** -* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape. One of the outputs corresponds to the output with execution condition. This output is marked in the port_map with attribute `purpose = "execution_condition"`. +* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape except the one corresponding to the output with execution condition. This output is marked in the port_map with attribute `purpose = "execution_condition"` and is mandatory and produces a scalar or 1D tensor with 1 element of `boolean` type. Other outputs are optional. **Examples**