Building Mxnet C++ on Centos Container Gives Error

Hello, I have been trying to build mxnet c++ library on “nvidia/cuda:9.2-cudnn7-devel-centos7” image. However it gives me the error at below.

(cd cpp-package/scripts; python OpWrapperGenerator.py /incubator-mxnet/lib/libmxnet.so)
Traceback (most recent call last):
  File "OpWrapperGenerator.py", line 428, in <module>
    raise(e)
OSError: libcuda.so.1: cannot open shared object file: No such file or directory
make: *** [cpp-package/include/mxnet-cpp/op.h] Error 1
make: *** Waiting for unfinished jobs....

When I search libcuda.so.1, it gives me these result.

$ find / -name libcuda.so.1
/usr/lib64/libcuda.so.1

While I cannot give the whole Dockerfile here is the mxnet build part.

ENV LD_LIBRARY_PATH=/usr/local/lib64/:/usr/local/lib/:/usr/local/cuda/lib64:/usr/lib64/:/mxnet/lib
RUN git clone --recursive https://github.com/apache/incubator-mxnet.git \
&& cd incubator-mxnet \
&& build_arg="USE_BLAS=openblas USE_CPP_PACKAGE=1 USE_LAPACK=1 USE_OPENCV=1" \
&& if [ $( which nvcc | wc -l ) -gt "0" ]; then build_arg+=" USE_CUDA=1 USE_CUDNN=1 USE$
make -j"$(nproc)" $build_arg 

When I try prebuilt python version, it works as it should.

Is using pip package not an option for you? Pip package is what’s used in SageMaker containers.

From what I know pip just install python apis. Here, For C++ it doesnt’t show a pip installation.

Right, didn’t really read your question carefully. You’re trying to compile for CPP package. Typically cuda is installed under /usr/local/cuda/ and *.so files for it are under /usr/local/cuda/lib64. Did you install cuda somewhere else?

I solved the problem. Create libcuda.so.1 symlink from cuda stub (For my case, it is at “/usr/local/cuda-9.2/targets/x86_64-linux/lib/stubs/libcuda.so”) and add that folder to LD_LIBRARY_PATH. After building both mxnet and the project that use mxnet, remove stub folder from LD_LIBRARY_PATH.

1 Like

Thanks for sharing your answer.