Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added specification for RandomUniform operation. #6196

Merged

Conversation

popovaan
Copy link
Contributor

@popovaan popovaan commented Jun 16, 2021

Details:

  • Specification of the operation RandomUniform.

Tickets:

  • 56591

Algorithm description

RandomUniform operation generates random numbers from a uniform distribution in the range [minval, maxval).
The generation algorithm is based on underlying random integer generator that uses Philox algorithm. Philox algorithm
is counter based pseudo random generator, which produces uint32 values. Single invocation of Philox algorithm returns
four result random values, depending on the given key and counter values. Key and counter are initialized
with seed and seed2 attributes respectively.

key = seed
counter = seed2

Link to the original paper [Parallel Random Numbers: As Easy as 1, 2, 3]
(https://www.thesalmons.org/john/random123/papers/random123sc11.pdf)

The result of Philox is calculated by applying a fixed number of key and counter updating, so-called "rounds".
This implementation uses 4x32_10 version of Philox algorithm, where number of rounds = 10.

Suppose we have n which determines n-th 4 elements of random sequence.
In each round key, counter and n are splitted to pairs of uint32 values:

f0

where cast_to_uint32 - static cast to uint32, value - uint64 input value, L, R - uint32
result values, >> - bitwise right shift.

Then n and counter are updated with the following formula:

f1

where oplus - bitwise xor, k = R_key for updating counter, k = L_key for updating n,
M = 0xD2511F53 for updating n, M = 0xCD9E8D57 for updating counter.

After each round key is raised by summing with another pair of const values:

L += 0x9E3779B9
R += 0xBB67AE85

Values L'_n, R'_n, L'_counter, R'_counter are resulting four random numbers.

Float values between [0..1) are obtained from 32-bit integers by the following rules.

Float16 is formatted as follows: sign(1 bit) exponent(5 bits) mantissa(10 bits). The value is interpreted
using following formula:

exp-15

so to obtain float16 values sign, exponent and mantissa are set as follows:

sign = 0
exponent = 15 - representation of a zero exponent.
mantissa = 10 right bits from generated uint32 random value.

So the resulting float16 value is:

x_uint16 = x // Truncate the upper 16 bits.
val = ((exponent << 10) | x_uint16 & 0x3ffu) - 1.0,

where x is uint32 generated random value.

Float32 is formatted as follows: sign(1 bit) exponent(8 bits) mantissa(23 bits). The value is interpreted
using following formula:

exp-127

so to obtain float values sign, exponent and mantissa are set as follows:

sign = 0
exponent = 127 - representation of a zero exponent.
mantissa = 23 right bits from generated uint32 random value.

So the resulting float value is:

val = ((exponent << 23) | x & 0x7fffffu) - 1.0,

where x is uint32 generated random value.

Double is formatted as follows: sign(1 bit) exponent(11 bits) mantissa(52 bits). The value is interpreted
using following formula:

exp-1023

so to obtain double values sign, exponent and mantissa are set as follows:

sign = 0
exponent = 1023 - representation of a zero exponent.
mantissa = 52 right bits from two concatenated uint32 values from random integer generator.

So the resulting double is obtained as follows:

mantissa_h = x0 & 0xfffffu;  // upper 20 bits of mantissa
mantissa_l = x1;             // lower 32 bits of mantissa
mantissa = (mantissa_h << 32) | mantissa_l;
val = ((exponent << 52) | mantissa) - 1.0,

where x0, x1 are uint32 generated random values.

To obtain a value in a specified range each value is processed with the following formulas:

For float values:
res_mul
where x is random float or double value between [0..1).

For integer values:
res_mod
where x is uint32 random value.

@popovaan popovaan requested a review from a team as a code owner June 16, 2021 11:35
@popovaan popovaan requested review from a team, pavel-esir, iimironov and vgavrilo and removed request for a team June 16, 2021 11:37
@openvino-pushbot openvino-pushbot added the category: docs OpenVINO documentation label Jun 16, 2021
Copy link
Contributor

@pavel-esir pavel-esir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one comment other looks good to me

@popovaan popovaan requested a review from vgavrilo June 16, 2021 14:38
@popovaan popovaan requested a review from vgavrilo June 17, 2021 14:16
@popovaan popovaan requested a review from vgavrilo June 17, 2021 14:36
@popovaan popovaan requested a review from lazarevevgeny June 18, 2021 12:17
@lazarevevgeny
Copy link
Contributor

Add plugin owners to the review

@popovaan popovaan requested review from esmirno and terfendail June 21, 2021 13:42
Copy link
Contributor

@lazarevevgeny lazarevevgeny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, add the rendered version of formulas to the PR description.

@popovaan popovaan requested a review from lazarevevgeny June 23, 2021 15:05
@popovaan popovaan requested a review from rkazants July 19, 2021 08:09
@popovaan popovaan requested a review from rkazants July 21, 2021 09:17
Copy link
Contributor

@lazarevevgeny lazarevevgeny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Let's pass the review from the technical writers and then merge the spec.

@andrew-zaytsev andrew-zaytsev requested a review from tsavina July 22, 2021 16:42
@popovaan popovaan requested a review from tsavina July 23, 2021 08:30
@lazarevevgeny lazarevevgeny merged commit c776ea9 into openvinotoolkit:master Jul 23, 2021
rnugmanx pushed a commit to rnugmanx/openvino that referenced this pull request Aug 26, 2021
* Added RandomUniform specification.

* Extended description of obtaining float numbers.

* Added description of obtainin float16 and double values.

* Added description of xor symbol.

* Small correction.

* Small corrections.

* Added initial type attribute.

* Corrected IR example, removed unnecessary default values.

* Small correction.

* Added information about type cast between initial and output types.

* Removed initial type attribute.

* Corrected output_type description.

* Corrected minval, maxval description. Corrected IR example.

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <[email protected]>

* Removed unnecessary paper link.

Co-authored-by: Tatiana Savina <[email protected]>
andrei-cv pushed a commit to andrei-cv/openvino that referenced this pull request Aug 30, 2021
* Added RandomUniform specification.

* Extended description of obtaining float numbers.

* Added description of obtainin float16 and double values.

* Added description of xor symbol.

* Small correction.

* Small corrections.

* Added initial type attribute.

* Corrected IR example, removed unnecessary default values.

* Small correction.

* Added information about type cast between initial and output types.

* Removed initial type attribute.

* Corrected output_type description.

* Corrected minval, maxval description. Corrected IR example.

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <[email protected]>

* Removed unnecessary paper link.

Co-authored-by: Tatiana Savina <[email protected]>
akuporos pushed a commit to akuporos/openvino that referenced this pull request Sep 29, 2021
* Added RandomUniform specification.

* Extended description of obtaining float numbers.

* Added description of obtainin float16 and double values.

* Added description of xor symbol.

* Small correction.

* Small corrections.

* Added initial type attribute.

* Corrected IR example, removed unnecessary default values.

* Small correction.

* Added information about type cast between initial and output types.

* Removed initial type attribute.

* Corrected output_type description.

* Corrected minval, maxval description. Corrected IR example.

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <[email protected]>

* Removed unnecessary paper link.

Co-authored-by: Tatiana Savina <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: docs OpenVINO documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants