Reduce memory footprint of nGraph

[rgesteve]

Description: CMake changes to properly configure ngraph build.

Motivation and Context
Memory use of the onnxruntime (using the ngraph EP) on Windows is higher than it is on Linux. The included change attempts to narrow this gap.

[pranavsharma]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 22 pipeline(s).

[pranavsharma]

Can you quantify how much improvement this patch achieves?

[skottmckay]

Do we need to patch or could we update the ngraph version we're using instead?

[jywu-msft]

Updating ngraph version is a bigger effort. I believe there are other issues with newer ngraph versions that are being tracked as separate issues.

[jywu-msft]

Remove email header?

[jfdong]

@jywu-msft we will remove the header.
@pranavsharma the impact of this fix is to avoid unnecessary memory allocation for 1x1 convolution. It is backported the existing MKL-DNN commit of oneapi-src/oneDNN@3b96d63.
For resnet-50 model (from https://s3.amazonaws.com/download.onnx/models/opset_9/resnet50.tar.gz). After applying this fix, the memory footprint on my CLX test machine is 1.12GB (release) and 1.13GB (Debug). Before that fix, it is about 3.72GB. My test machine is Cascade Lake server.

@skottmckay I agree with @jywu-msft , there is bigger effort of nGraph update. This patch will help during the interim.

[pranavsharma]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 22 pipeline(s).

[sreekanth-yalachigere]

@jywu-msft @pranavsharma
This MKL-DNN patch for nGraph also reduces MKL-DNN EP memory usage significantly. Thanks

[pranavsharma]

@jywu-msft not merging this yet in case you want to run some tests or have other concerns.

[jywu-msft]

@jdfong thanks for the CLX numbers. Would it be possible for you to verify and provide expected numbers on AVX2 (on older system such as Haswell/Broadwell) One if our devs wasn't able to repro the resnet50 memory behavior on his system with AVX2.
Memory consumption was 1GB with or without the fix.

[jywu-msft]

@sreekanth-yalachigere re: your comment about mkl-dnn EP mem usage,
This PR only patches mkl-dnn used by ngraph right?
Wouldn't you need to apply same patch for mkl-dnn ep? Or upgrade mkl-dnn version...

[jywu-msft]

these changes look. good. we should look at further reducing memory footprint. goal is to be as close to default cpu provider as possible.