MXNet Build from source Mac - opencv not found

#1

Trying to build mxnet from source
Mac Mojave
make -j8
fails with

Makefile:187: "USE_LAPACK disabled because libraries were not found"
Makefile:313: WARNING: Significant performance increases can be achieved by installing and enabling gperftools or jemalloc development packages
Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv.pc'
to the PKG_CONFIG_PATH environment v

Followed by multiple occurrences of

In file included from src/operator/contrib/adaptive_avg_pooling.cc:25:
In file included from src/operator/contrib/adaptive_avg_pooling-inl.h:30:
In file included from include/mxnet/operator.h:38:
In file included from include/mxnet/./base.h:33:
In file included from /Users/chaitanyabapat/workspace/incubator-mxnet/3rdparty/mshadow/mshadow/tensor.h:16:
/Users/chaitanyabapat/workspace/incubator-mxnet/3rdparty/mshadow/mshadow/./base.h:162:14: fatal error: 'cblas.h' file not found
    #include <cblas.h>
             ^~~~~~~~~

Tried to resolve the blas issue -
By changing from apple to openblas
after installing brew install openblas

Still the error persists

It looks like for some reason, brew install packages are not accesible.
That’s the reason why cblas as well as opencv errors are thrown

$ pkg-config --libs openblas
Package openblas was not found in the pkg-config search path.
Perhaps you should add the directory containing `openblas.pc'
to the PKG_CONFIG_PATH environment variable
No package 'openblas' found
$ pkg-config --libs opencv
Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv.pc'
to the PKG_CONFIG_PATH environment variable
No package 'opencv' found

Googling above error on stackoverflow gives me solution of
updating environment variable; that doesn’t work either

$ vi ~/.bashrc 
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/lib

$ source ~/.bashrc

$ echo $PKG_CONFIG_PATH
/usr/local/lib/pkgconfig:/usr/local/lib

Out of options, please help.
Thanks

#2

Did you follow the instructions from here: https://mxnet.incubator.apache.org/versions/master/install/osx_setup.html ?
Can you check where your openblas headers and library are located? If they are in a non-standard path, then you need to add this path in some environment variables e.g. PATH, INCLUDE
When building MXNet from source, it is maybe easier to try using CMake. With the command ccmake you can set all the necessary environment variables.

#3

Yes I followed the steps mentioned there and that’s where the issues come form.

#4

Can you try using CMake?
You can install it with brew install cmake. Then execute ccmake . in MXNet src-directory. You need to execute configure, which loads all CMakeOptions. You are then able to set the required paths for OpenCV and OpenBlas. Once all options are set, you can generate the Makefile.

#5

Firstly,
ccmake . can’t be run in \src directory of MXNet.

src chaitanyabapat$ ccmake .

CMake Error: The source directory "/Users/chaitanyabapat/workspace/incubator-mxnet/src" does not appear to contain CMakeLists.txt.

Specify --help for usage, or press the help button on the CMake GUI.

I tried running it in the base directory of incubator-mxnet, command atleast ran. I was able to see the loaded CMakeOptions.

a. Where can I set the path for OpenCV and OpenBlas?
b. If I try to press [c] for configure at USE_OPENCV ON - it gives me the following error

CMake Error at CMakeLists.txt:71 (project):
No CMAKE_CUDA_COMPILER could be found.

   Tell CMake where to find the compiler by setting either the environment
   variable "CUDACXX" or the CMake cache entry CMAKE_CUDA_COMPILER to the full
   path to the compiler, or to the compiler name if it is in the PATH.

More importantly, I am just curious - I was able to build and code and contribute to MXNet for all these months (Aug - Dec 2018) on other Mac laptop. With all steps remaining the same, only machine changed - Why am I getting these errors? What’s going wrong? There’s some real weird mistake happening.

#6

Sorry I meant MXNet’s base-directory not src. I assume you don’t have CUDA installed? If so, you need to set USE_CUDA=OFF. If CMake gives errors due to missing dependencies: you can either disable the corresponding CMake option or install the missing library.
In order to configure CMake options you can just scroll through the menu, in order to modify a certain item press enter and you can type the corresponding path. Once you have adjusted all paths, you can press [c]: this loads all the options and checks the dependencies. Alternatively you can just directly modify the file CMakeCache.txt: all CMake options are stored in there and ccmake uses that file to generate the final Makefile.
The CMake option OpenCV_DIR defines the path to OpenCV library.

#7

Turning CUDA OFF didn’t help solve the blas.h not found issue.
Still stuck.

#8

My guess is that either brew install openblas did not install header files or it did install them in a non-standard location. I don’t have a Mac with Mojave, so I can’t verify if this is the problem. In Sierra brew install openblas installs headerfiles under /usr/local/opt/openblas/include.

#9

Ya, that’s what even my primary guess was. But as it turns out the header files are installed in the same location as you mentioned

$ pwd
/usr/local/opt/openblas/include
$ ls
cblas.h			f77blas.h		lapacke.h		lapacke_config.h	lapacke_mangling.h	lapacke_utils.h		openblas_config.h

I’m starting to wonder if its coz me upgrading to Mojave. Unable to build and my progress is stalled.

#10

I had to do something similar to this when I setup my new laptop https://silvae86.github.io/sysadmin/mac/osx/mojave/beta/libxml2/2018/07/05/fixing-missing-headers-for-homebrew-in-mac-osx-mojave/

#11

Thanks a lot. Worked!