Default build with CMake not using MKL

[larroy]

With make MKL is installed automatically, but with CMake in linux not.

piotr@ip-172-31-30-72:0:~/mxnet (master)$ cp cmake/cmake_options.yml .
piotr@ip-172-31-30-72:0:~/mxnet (master)$ grep MKL cmake_options.yml
USE_MKL_IF_AVAILABLE: "ON" # Use MKL if found                                                                                                                                                                                                USE_MKLML_MKL: "ON" # Use MKLDNN variant of MKL (if MKL found) IF USE_MKL_IF_AVAILABLE AND (NOT APPLE)                                                                                                                                       USE_MKLDNN: "ON" # Use MKLDNN variant of MKL (if MKL found) IF USE_MKL_IF_AVAILABLE AND (NOT APPLE)                                                                                                                                          piotr@ip-172-31-30-72:0:~/mxnet (master)$ ./dev_menu.py menu 1 | tee build.log
INFO:root:CMake / Ninja build in directory /home/piotr/mxnet/build
INFO:root:Executing: cmake
-DUSE_CUDA=OFF
-DUSE_OLDCMAKECUDA=OFF
-DUSE_NCCL=OFF
-DUSE_OPENCV=ON
-DUSE_OPENMP=ON
-DUSE_CUDNN=ON
-DUSE_SSE=ON
-DUSE_F16C=ON
-DUSE_LAPACK=ON
-DUSE_MKL_IF_AVAILABLE=ON
-DUSE_MKLML_MKL=ON
-DUSE_MKLDNN=ON
-DUSE_OPERATOR_TUNING=ON
-DUSE_GPERFTOOLS=ON
-DUSE_JEMALLOC=ON
-DUSE_PROFILER=ON
-DUSE_DIST_KVSTORE=OFF
-DUSE_PLUGINS_WARPCTC=OFF
-DUSE_PLUGIN_CAFFE=OFF
-DUSE_CPP_PACKAGE=OFF
-DUSE_MXNET_LIB_NAMING=ON
-DUSE_GPROF=OFF
-DUSE_CXX14_IF_AVAILABLE=OFF
-DUSE_VTUNE=OFF
-DENABLE_CUDA_RTC=ON
-DBUILD_CPP_EXAMPLES=ON
-DINSTALL_EXAMPLES=OFF
-DUSE_SIGNAL_HANDLER=ON
-DUSE_TENSORRT=OFF
-DUSE_ASAN=OFF
-DENABLE_TESTCOVERAGE=OFF
-DCMAKE_BUILD_TYPE=Debug
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-GNinja
..
-- CMAKE_CROSSCOMPILING FALSE
-- CMAKE_HOST_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_NAME Linux
-- Determining F16C support
-- Downloading MKLML...
CMake Error at cmake/DownloadMKLML.cmake:66 (file):
  file DOWNLOAD HASH mismatch

    for file: [/home/piotr/mxnet/build/mklml/mklml_lnx_2019.0.3.20190220.tgz]
      expected hash: [76354b74325cd293aba593d7cbe36b3f]
        actual hash: [d41d8cd98f00b204e9800998ecf8427e]
             status: [1;"Unsupported protocol"]

Call Stack (most recent call first):
  CMakeLists.txt:242 (include)




gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
-- Setting MKLROOT path to /home/piotr/mxnet/build/mklml/mklml_lnx_2019.0.3.20190220
-- This is a product build
-- Detecting Intel(R) MKL: trying mklml_intel
-- Detecting Intel(R) MKL: trying mklml_gnu
-- Detecting Intel(R) MKL: trying mklml
-- Detecting Intel(R) MKL: trying mkl_rt
CMake Warning at 3rdparty/mkldnn/cmake/MKL.cmake:267 (message):
  Intel(R) MKL not found.  Some performance features may not be available.
  Please run scripts/prepare_mkl.sh to download a minimal set of libraries or
  get a full version from https://software.intel.com/en-us/intel-mkl
Call Stack (most recent call first):
  3rdparty/mkldnn/cmake/OpenMP.cmake:25 (include)
  3rdparty/mkldnn/CMakeLists.txt:76 (include)


-- Found OpenMP_C: -fopenmp
-- Found OpenMP_CXX: -fopenmp
-- OpenMP lib: provided by compiler
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- VTune profiling environment is unset
CMake Warning (dev) at cmake/ChooseBlas.cmake:23 (find_package):
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  CMake variable MKL_ROOT is set to:

    MKL_ROOT-NOTFOUND

  For compatibility, CMake is ignoring the variable.
Call Stack (most recent call first):
  CMakeLists.txt:298 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- Found OpenBLAS libraries: /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Found OpenBLAS include: /usr/include/x86_64-linux-gnu
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- Found OpenBLAS libraries: /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Found OpenBLAS include: /usr/include/x86_64-linux-gnu
-- Could NOT find Gperftools (missing: GPERFTOOLS_LIBRARIES GPERFTOOLS_INCLUDE_DIR)
-- Using JEMalloc malloc
--  OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc;opencv_imgcodecs
-- OpenCV 3.2.0 found (/usr/share/OpenCV)
-- LIBOMP: Operating System     -- Linux
-- LIBOMP: Target Architecture  -- x86_64
-- LIBOMP: Build Type           -- Debug
-- LIBOMP: OpenMP Version       -- 50
-- LIBOMP: Library Kind         -- SHARED
-- LIBOMP: Library Type         -- normal
-- LIBOMP: Fortran Modules      -- FALSE
-- LIBOMP: Build                -- 20140926
-- LIBOMP: Use Stats-gathering  -- FALSE
-- LIBOMP: Use Debugger-support -- FALSE
-- LIBOMP: Use ITT notify       -- TRUE
-- LIBOMP: Use OMPT-support     -- FALSE
-- LIBOMP: Use Adaptive locks   -- TRUE
-- LIBOMP: Use quad precision   -- TRUE
-- LIBOMP: Use TSAN-support     -- FALSE
-- LIBOMP: Use Hwloc library    -- FALSE
-- LIBOMP: Cannot find llvm-lit.

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Build

[pengzhao-intel]

thank to report this issue @larroy

@NeoZhangJianyu @yinghu5 is working on the CMake for MKL

[larroy]

@mxnet-label-bot add [CMake, Build, MKL]

[NeoZhangJianyu]

@larroy I'm fixing this issue now.
There are several problems for this issue:

1. $MKLROOT is set by MKL installation, but this cmake expect $MKL_ROOT.
2. If MKLDNN is used, it will use OpenBLAS, instead of MKL as BLAS.
3. Wrong dependence check in MKLDNN and MKL. It will make the MKL check is fault.

I have fixed it in linux locally. But the solution need to be reviewed.
And we need to fix it in windows build too. (The cmakefile support linux, windows and mac in same time)
So it will take more days to be verified and committed.

[pengzhao-intel]

cc @TaoLv

[larroy]

Thanks for the great response guys, you rock!

[sl1pkn07]

@NeoZhangJianyu can you also fix this?

CMake Warning (dev) at 3rdparty/mkldnn/cmake/utils.cmake:120 (target_link_libraries):
  Policy CMP0023 is not set: Plain and keyword target_link_libraries
  signatures cannot be mixed.  Run "cmake --help-policy CMP0023" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  The plain signature for target_link_libraries has already been used with
  the target "mkldnn".  All uses of target_link_libraries with a target
  should be either all-keyword or all-plain.

  The uses of the plain signature are here:

   * 3rdparty/mkldnn/cmake/utils.cmake:111 (target_link_libraries)

Call Stack (most recent call first):
  3rdparty/mkldnn/src/CMakeLists.txt:108 (target_link_libraries_install)
This warning is for project developers.  Use -Wno-dev to suppress it.

with cmake 3.14.2

greetings

[NeoZhangJianyu]

@sl1pkn07
It's a warning, you could ignore it.

Or fix it manually:

Edit 3rdparty/mkldnn/cmake/utils.cmake:111:

From

function(target_link_libraries_build target list)
    # Foreach is required for compatibility with 2.8.11 ways
    foreach(lib ${list})
            target_link_libraries(${target}
            "$<BUILD_INTERFACE:${lib}>")
    endforeach(lib)
endfunction()

to

function(target_link_libraries_build target list)
    # Foreach is required for compatibility with 2.8.11 ways
    foreach(lib ${list})
            target_link_libraries(${target} PUBLIC
            "$<BUILD_INTERFACE:${lib}>")
    endforeach(lib)
endfunction()

[sl1pkn07]

yep. but should be "fixed"/silenced in upstream, or i'am wrong?

greetings