I am trying to contribute code to MxNet, but whenever I get time, I find myself trying to get the build system to work. It there a good way to get consistent builds? Like a Dockerfile that builds my current source tree.
Hi @aidan-plenert-macdon, Which part of the build are you having issues with? Are some of the unit tests failing?
No just running make fails with a variety of errors depending on the day. I just run on the master branch and it fails. I know this doesn’t help you help me debug the build, but I am hoping for a better way to build. Currently I am just running
make and it fails, so I have to go and fiddle with the various submods till the build gets a little further.
you can build your current source tree in a docker container with all the necessary dependencies by running this for example, for a GPU cuda9.1-cuddn7 build:
ci/build.py --platform ubuntu_build_cuda /work/runtime_functions.sh build_ubuntu_gpu_cuda91_cudnn7
Have a look at the
ci folder and the README, you have a few build platforms available.
Also remember when checking out mxnet to use
git clone https://github.com/apache/incubator-mxnet --recursive to get the submodule checked out as well.
@ThomasDelteil So that works in that it runs, but once it finishes and I get dropped into the container, neither python2 nor python3 have mxnet installed and further more, make still fails
$ ci/build.py -p ubuntu_cpu --into-container into container: True build.py: 2018-04-23 16:18:27,613 Building container tagged 'mxnet/build.ubuntu_cpu' with docker build.py: 2018-04-23 16:18:27,614 Running command: 'docker build -f docker/Dockerfile.build.ubuntu_cpu --build-arg USER_ID=514839664 -t mxnet/build.ubuntu_cpu docker' Sending build context to Docker daemon 112.1kB ... 28 steps later ... Successfully built 43573ca2ee85 Successfully tagged mxnet/build.ubuntu_cpu:latest groups: cannot find name for group ID 1896053708 jenkins_slave@b8c235a1940a:/work/mxnet$ python3 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import mxnet Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named 'mxnet' >>> jenkins_slave@b8c235a1940a:/work/mxnet$ make Makefile:240: WARNING: Significant performance increases can be achieved by installing and enabling gperftools or jemalloc development packages make: *** No rule to make target '~/incubator-mxnet/3rdparty/dmlc-core/include/dmlc/omp.h', needed by 'build/src/operator/nn/softmax.o'. Stop.
For the right make flags, follow the instructions of the build from source page tab:
From what I see you haven’t cloned the github repo properly, as in it is missing the submodules.
try to run
git submodule update --init --recursive
make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas
For python, you can check the instructions on the install page to add it to your python installation or just use
export PYTHONPATH=/work/mxnet/python before launching your python interpreter or notebook.
git clone https://github.com/apache/incubator-mxnet --recursive mxnet cd mxnet ci/build.py -p ubuntu_cpu --into-container make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas
successfully builds mxnet CPU version