replacing windows setup with newer instructions #12504

aaronmarkham · 2018-09-10T20:56:44Z

Description

When merging the different Windows guides I discarded some newer instructions. I'm bringing them back with this PR.

I also added a ToC and clarified some headings.

I'm leaving this as a WIP PR since I hope to get some feedback from people trying out the instructions and validating the steps.

aaronmarkham · 2018-09-10T20:57:35Z

@samskalicky @ankkhedia - please provide any feedback you can from your experiments with the guide. Thanks!

aaronmarkham · 2018-09-12T17:49:34Z

docs/install/windows_setup.md

+3. Download and install [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download).
+4. Unzip the OpenCV package.
+5. Set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g., ```OpenCV_DIR = C:\utils\opencv\build```).
+6. If you don’t have the Intel Math Kernel Library (MKL) installed, download and install [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.20/OpenBLAS%200.2.20%20version.zip/download).


Use this instead:
https://sourceforge.net/projects/openblas/files/v0.2.19/OpenBLAS-v0.2.19-Win64-int32.zip/download

This contains the dll which can be directly used else you will need to build openblas from source on Windows as no latest pre-built binaries are available after that version-not that I know of!

aaronmarkham · 2018-09-12T17:51:52Z

docs/install/windows_setup.md

+6. If you don’t have the Intel Math Kernel Library (MKL) installed, download and install [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.20/OpenBLAS%200.2.20%20version.zip/download).
+7. Set the environment variable ```OpenBLAS_HOME``` to point to the ```OpenBLAS``` directory that contains the ```include``` and ```lib``` directories (e.g., ```OpenBLAS_HOME = C:\utils\OpenBLAS```).
+8. Download and install CUDA: Install [CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal), and Download the base installer (e.g., ```cuda_9.1.85_win10.exe```).
+9. Download and install cuDNN. To get access to the download link, register as an NVIDIA community user. Then Follow the [link](http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#install-windows) to install the cuDNN.


Even if you have already installed CUDA, you should reinstall to get VS toolkit support if it isn't also already installed.

This is required since CUDA require VS integration support when it is being installed so that VS can be used to build binaries with CUDA support

aaronmarkham · 2018-09-12T17:55:41Z

docs/install/windows_setup.md


-After you have installed all of the required dependencies, build the MXNet source code:
+3. Follow [this link](https://docs.microsoft.com/en-us/visualstudio/install/modify-visual-studio) to modify ```Individual components```, and check ```VC++ 2017 version 15.4 v14.11 toolset```, and click ```Modify```.


Move this step to just after VS install.

and before CUDA toolkit install

aaronmarkham · 2018-09-12T17:58:06Z

docs/install/windows_setup.md


-After you have installed all of the required dependencies, build the MXNet source code:
+3. Follow [this link](https://docs.microsoft.com/en-us/visualstudio/install/modify-visual-studio) to modify ```Individual components```, and check ```VC++ 2017 version 15.4 v14.11 toolset```, and click ```Modify```.
+4. Change the version of the Visual studio 2017 to v14.11 using the following command (by default the VS2017 is installed in the following path):


This should be changed to setting on the command prompt after vs install.

Specifically this is needed for 1. 'cmake...' instruction and 2. 'msbuild..' instruction mentioned below.

aaronmarkham · 2018-09-12T18:01:30Z

docs/install/windows_setup.md

+6. CMake the MXNet source code by using following command:
+
+```
+cmake -G "Visual Studio 15 2017 Win64" -T cuda=9.1,host=x64 -DUSE_CUDA=1 -DUSE_CUDNN=1 -DUSE_NVRTC=1 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_LIST=Common -DCUDA_TOOLSET=9.1 -DCUDNN_INCLUDE=C:\cuda\include -DCUDNN_LIBRARY=C:\cuda\lib\x64\cudnn.lib "C:\incubator-mxnet"


alternative commands for MKL MKLDNN...
https://github.com/apache/incubator-mxnet/blob/547c9bad7915620f785375ca4921c377c0dc07b6/docs/install/ubuntu_setup.md

aaronmarkham · 2018-09-12T18:02:59Z

docs/install/windows_setup.md

+6. CMake the MXNet source code by using following command:
+
+```
+cmake -G "Visual Studio 15 2017 Win64" -T cuda=9.1,host=x64 -DUSE_CUDA=1 -DUSE_CUDNN=1 -DUSE_NVRTC=1 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_LIST=Common -DCUDA_TOOLSET=9.1 -DCUDNN_INCLUDE=C:\cuda\include -DCUDNN_LIBRARY=C:\cuda\lib\x64\cudnn.lib "C:\incubator-mxnet"


also show cuda=9.2 and 9.0
and -DCUDA_TOOLSET=9.2

Need to add a list of flag options available

aaronmarkham · 2018-09-12T18:05:35Z

docs/install/windows_setup.md


+NOTE: make sure the DCUDNN_INCLUDE and DCUDNN_LIBRARY pointing to the “include” and “cudnn.lib” of your CUDA installed location, and the ```C:\incubator-mxnet``` is the location of the source code you just git in the previous step


update grammar

aaronmarkham · 2018-09-12T18:07:03Z

docs/install/windows_setup.md

-&nbsp;
-This produces a library called ```libmxnet.dll```.
+1. If [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/) is not already installed, download and install it. You can download and install the free community edition.
+2. Download and install [CMake](https://cmake.org/files/v3.11/cmake-3.11.0-rc4-win64-x64.msi) if it is not already installed.


There is a later version that isn't tested.

aaronmarkham · 2018-09-12T18:07:17Z

docs/install/windows_setup.md

-This produces a library called ```libmxnet.dll```.
+1. If [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/) is not already installed, download and install it. You can download and install the free community edition.
+2. Download and install [CMake](https://cmake.org/files/v3.11/cmake-3.11.0-rc4-win64-x64.msi) if it is not already installed.
+3. Download and install [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download).


There is a later version that isn't tested.

aaronmarkham · 2018-09-12T18:07:34Z

docs/install/windows_setup.md

+3. Download and install [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download).
+4. Unzip the OpenCV package.
+5. Set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g., ```OpenCV_DIR = C:\utils\opencv\build```).
+6. If you don’t have the Intel Math Kernel Library (MKL) installed, download and install [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.20/OpenBLAS%200.2.20%20version.zip/download).


There are later versions but no binaries available.

aaronmarkham · 2018-09-12T18:08:12Z

docs/install/windows_setup.md

+4. Unzip the OpenCV package.
+5. Set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g., ```OpenCV_DIR = C:\utils\opencv\build```).
+6. If you don’t have the Intel Math Kernel Library (MKL) installed, download and install [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.20/OpenBLAS%200.2.20%20version.zip/download).
+7. Set the environment variable ```OpenBLAS_HOME``` to point to the ```OpenBLAS``` directory that contains the ```include``` and ```lib``` directories (e.g., ```OpenBLAS_HOME = C:\utils\OpenBLAS```).


Point to MKL setup instructions.

codecov-io · 2018-09-12T22:31:36Z

Codecov Report

Merging #12504 into master will decrease coverage by 0.07%.
The diff coverage is 81.91%.

@@            Coverage Diff             @@
##           master   #12504      +/-   ##
==========================================
- Coverage   72.48%    72.4%   -0.08%     
==========================================
  Files         163      161       -2     
  Lines       19165    19131      -34     
  Branches     3136     3134       -2     
==========================================
- Hits        13891    13852      -39     
- Misses       4403     4409       +6     
+ Partials      871      870       -1

Impacted Files	Coverage Δ
python/mxnet/image/detection.py	`61.61% <0%> (-1.44%)`	⬇️
python/mxnet/io.py	`81.67% <92.77%> (ø)`
python/mxnet/ndarray/sparse.py	`74.78% <0%> (+0.42%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46a5cee...9196901. Read the comment docs.

aaronmarkham · 2018-09-13T15:30:33Z

@ankkhedia WDYT about the current changes? Should we go ahead and push these so better info is out there? We can make another pass on it later... or if possible, I'd like to update the VS2015 today and then push.

ankkhedia · 2018-09-13T16:47:50Z

@aaronmarkham The changes for vs2017 looks good to me and worked for me. However, it would make sense to ask someone to try it out in case there is some system info which might be missing. For VS2015, could you please add CI owners to the review as they build Windows for CI and they can validate it.

Anyways I am fine with pushing these changes as as this is better info from current

kalyc · 2018-09-14T17:54:27Z

Thanks for your contributions @aaronmarkham
@mxnet-label-bot[pr-work-in-progress]

aaronmarkham · 2018-09-15T00:02:02Z

@vrakesh - Please let me know your feedback/improvements! Thanks!

vrakesh · 2018-09-15T00:19:16Z

We should mention, the requirement of Visual Studio (needed by CUDA) and also link to the detailed CUDA installation instructions on NVIDIA website.

yajiedesign · 2018-09-16T03:23:13Z

docs/install/windows_setup.md

+cmake -G "Visual Studio 15 2017 Win64" -T cuda=9.2,host=x64 -DUSE_CUDA=1 -DUSE_CUDNN=1 -DUSE_NVRTC=1 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_LIST=Common -DCUDA_TOOLSET=9.2 -DCUDNN_INCLUDE=C:\cuda\include -DCUDNN_LIBRARY=C:\cuda\lib\x64\cudnn.lib "C:\incubator-mxnet"
+```
+6. After the CMake successfully completed, compile the the MXNet source code by using following command:
+```


It is recommended to add a section.the cmake 3.12.2 can use -T version=14.11 to choice vs14.11.

@yajiedesign -
I'm updating the CMake dependency to be 3.12.2.

For the cmake command, that would choose Visual Studio 2015's compiler? If we're trying to use 2017's compiler wouldn't this command need to be different? I don't understand this mix if I use -T version=14.11 in that cmake command:

cmake -G "Visual Studio 15 2017 Win64" -T version=14.11,cuda=9.2,host=x64

Can you be very specific about the command to use for each environment?

if you use -T version=14.11,you no need for pre execution "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat" -vcvars_ver=14.11

…a, windows pip note

aaronmarkham · 2018-09-17T18:49:41Z

Updated from feedback... this should be merged once it passes CI.

* replacing windows setup with newer instructions * add pip instructions; clarify 2017 build * refinements * bump cmake version & note compiler option; add vs requirement for cuda, windows pip note

replacing windows setup with newer instructions

b9dbce9

aaronmarkham requested a review from szha as a code owner September 10, 2018 20:56

Merge branch 'master' into windows_instructions

be2587c

aaronmarkham commented Sep 12, 2018

View reviewed changes

add pip instructions; clarify 2017 build

9196901

marcoabreu added the pr-work-in-progress PR is still work in progress label Sep 14, 2018

refinements

e8407cf

yajiedesign reviewed Sep 16, 2018

View reviewed changes

yajiedesign mentioned this pull request Sep 16, 2018

[MXNET-955]add install doc with vs2017 #12573

Closed

2 tasks

bump cmake version & note compiler option; add vs requirement for cud…

055ea4e

…a, windows pip note

aaronmarkham changed the title ~~[WIP] replacing windows setup with newer instructions~~ replacing windows setup with newer instructions Sep 17, 2018

indhub merged commit d8c51e5 into apache:master Sep 17, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

replacing windows setup with newer instructions #12504

replacing windows setup with newer instructions #12504

aaronmarkham commented Sep 10, 2018

aaronmarkham commented Sep 10, 2018

aaronmarkham Sep 12, 2018

ankkhedia Sep 12, 2018

aaronmarkham Sep 12, 2018 •

edited

Loading

ankkhedia Sep 12, 2018

aaronmarkham Sep 12, 2018

ankkhedia Sep 12, 2018

aaronmarkham Sep 12, 2018

ankkhedia Sep 12, 2018

aaronmarkham Sep 12, 2018

aaronmarkham Sep 12, 2018 •

edited

Loading

ankkhedia Sep 12, 2018

aaronmarkham Sep 12, 2018

aaronmarkham Sep 12, 2018

aaronmarkham Sep 12, 2018

aaronmarkham Sep 12, 2018

aaronmarkham Sep 12, 2018

codecov-io commented Sep 12, 2018 •

edited

Loading

aaronmarkham commented Sep 13, 2018

ankkhedia commented Sep 13, 2018 •

edited

Loading

kalyc commented Sep 14, 2018

aaronmarkham commented Sep 15, 2018

vrakesh commented Sep 15, 2018

yajiedesign Sep 16, 2018

aaronmarkham Sep 17, 2018

yajiedesign Sep 18, 2018

aaronmarkham commented Sep 17, 2018


		After you have installed all of the required dependencies, build the MXNet source code:
		3. Follow [this link](https://docs.microsoft.com/en-us/visualstudio/install/modify-visual-studio) to modify ```Individual components```, and check ```VC++ 2017 version 15.4 v14.11 toolset```, and click ```Modify```.


		NOTE: make sure the DCUDNN_INCLUDE and DCUDNN_LIBRARY pointing to the “include” and “cudnn.lib” of your CUDA installed location, and the ```C:\incubator-mxnet``` is the location of the source code you just git in the previous step

replacing windows setup with newer instructions #12504

replacing windows setup with newer instructions #12504

Conversation

aaronmarkham commented Sep 10, 2018

Description

aaronmarkham commented Sep 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronmarkham Sep 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronmarkham Sep 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Sep 12, 2018 • edited Loading

Codecov Report

aaronmarkham commented Sep 13, 2018

ankkhedia commented Sep 13, 2018 • edited Loading

kalyc commented Sep 14, 2018

aaronmarkham commented Sep 15, 2018

vrakesh commented Sep 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronmarkham commented Sep 17, 2018

aaronmarkham Sep 12, 2018 •

edited

Loading

aaronmarkham Sep 12, 2018 •

edited

Loading

codecov-io commented Sep 12, 2018 •

edited

Loading

ankkhedia commented Sep 13, 2018 •

edited

Loading