[Explanation] VPU_NUMBER_OF_CMX_SLICES

[franva]

System information (version)

• OpenVINO => 2021.4
• Operating System / Platform => Ubuntu 20.04

Detailed description

Trying to use the compile_tool to compile my IR model.
Would like to specify how many shaves the model should use, but found that the VPU_NUMBER_OF_SHAVES needs to be used together with VPU_NUMBER_OF_CMX_SLICES.

The problem is that the VPU_NUMBER_OF_CMX_SLICES is not given a range, so we do not know what numbers we can put there. After a few rounds of try-and-error, I found it's value range, but I have no idea what's the correlation with VPU_NUMBER_OF_SHAVES. E.g. when I set shave=8, I should also set the slice=8, or there is a formula to use to calculate the correct number of VPU_NUMBER_OF_CMX_SLICES.

It will be nice if the document can be more detailed with the range and it's related to the number of shave.

[zulkifli-halim]

Hello @franva, based on the Compile Tool Documentation , specifies the number of CMX slices, should be set with "VPU_NUMBER_OF_SHAVES".

[franva]

@zulkifli-halim
Thanks for your reply.

I think they do not have similar values.

Shave ranges from 1 to 16, whereas the CMX slices has a upper limit of 19.

Also, I would like to know how to calculate that when I set X shaves, how many Y CMS slices should I use.

And what does this VPU_NUMBER_OF_SLICES really do for me?

[zulkifli-halim]

Hi @franva, We don't have any other documentation for compile tool. The pull request #2946 and information from this code might help to understand it better.

[coneypo]

The Connection Matrix (CMX) enables seamless interaction between the vector processors and other hardware components;

Example 1.

CMX memory = 2 MB，considered as 16x128 KB ‘slices’, and 12 shaves on this Myriad device;

Shave	Slice	Start Address	End Address	Size
0	0	0x70000000	0x7001FFFF	128 KB
1	1	0x70020000	0x7003FFFF	128 KB
2	2	0x70040000	0x7005FFFF	128 KB
3	3	0x70060000	0x7007FFFF	128 KB
4	4	0x70080000	0x7009FFFF	128 KB
5	5	0x700A0000	0x700BFFFF	128 KB
6	6	0x700C0000	0x700DFFFF	128 KB
7	7	0x700E0000	0x700FFFFF	128 KB
8	8	0x70100000	0x7011FFFF	128 KB
9	9	0x70120000	0x7013FFFF	128 KB
10	10	0x70140000	0x7015FFFF	128 KB
11	11	0x70160000	0x7017FFFF	128 KB
x	12	0x70180000	0x7019FFFF	128 KB
x	13	0x701A0000	0x701BFFFF	128 KB
x	14	0x701C0000	0x701DFFFF	128 KB
x	15	0x701E0000	0x701FFFFF	128 KB

• Shave 0 is assgined the lowest 128 Kbyte of CMX (Slice 0), Shave 1 to Slice 1... Shave 11 to Slice 11;
• Slices 12/13/14/15 are not tied to any Shave, freely used;
• Each SHAVE can access any other slice in CMX at the same cost, but inter-slice routing resources are finite. In addition, a slave accessing data in its own slice is more energy-efficient.

So for this device, shave ranges from 1 to 12, and CMX slices has a upper limt of 16;

Example 2.

CMX memory = 1 MB, considered as 8x128 KB slices, and one 128 KB CMX slice for one Shave:
So for this device, shave ranges from 1 to 8, and CMX slice also range from 1 to 8;

So VPU_NUMBER_OF_SHAVES and VPU_NUMBER_OF_CMX_SLICES both are the capabilities of HW Myriad device;

BTW, in this MR #6022, max numShaves is locked to 16 and max numSlices is locked to 19 in OV now:

int DeviceResources::numShaves() {
    return 16;
}

int DeviceResources::numSlices() {
    return 19;
}

Please refer to:

• Application processor description from Movidius team: https://ec.europa.eu/research/participants/documents/downloadPublic?documentIds=080166e5a6e0e311&appId=PPGMS, which you can find introductions for SHAVE and CMX for myriad;
• https://en.wikichip.org/wiki/movidius/microarchitectures/shave_v2.0

[zulkifli-halim]

Closing issue.

[franva]

thank @coneypo for your detailed and picture-filled answer.

For both of your examples, you knew the CMX size at the beginning e.g. 2MB for the 1st example and 1MB for the 2nd example.
and you also knew number of "slices" already.

For an user, how can I know how big is my CMX and how many "slices" are there in my Intel's hardware, e.g. Myriad X VPU?

Are shaves the vector processors?

I will read through the links you provided.

Once again, thank you for providing the information :)