Strange compile errors

I am trying to build mxnet from source and I get a couple of strange errors.

The first error is this

Running: OpWrapperGenerator.py
argument "begin" of operator "slice" has unknown type "tuple of <>, required"
argument "end" of operator "slice" has unknown type "tuple of <>, required"
argument "step" of operator "slice" has unknown type "tuple of <>, optional, default=[]"
argument "end" of operator "slice_axis" has unknown type ", required"

This then results in the tests and the cpp examples failing to build as the function signature for slice doesn’t have any types for the begin and end parameters.

The second error occurs when trying to build the cython modules

/usr/bin/cmake -E env MXNET_LIBRARY_PATH=/home/bidski/Projects/mxnet/build/libmxnet.so 
/usr/bin/python3 setup.py build_ext --inplace --with-cython
free(): invalid pointer
Child aborted

The error seems to occur when on this line of python/setup.py. Which kicks off a massive chain of imports in python/mxnet, but I have yet to track down exactly what it causing the error. My assumption is that somewhere along the line, the mxnet library is being loaded and that is causing the error, but as i haven’t been able to track down the exact source I have nothing to back this assumption up.

I have made some changes to the CMake build system in an attempt to resolve some of other build issues I have been having, so I am not sure if I have messed something up in the build system or if this is some other issue that I just haven’t seen before.

Has anyone ever seen either of these issues before, or have I screwed something up mightily?

Do you have to build with CMake? If not, you can try Make.

Even compiling with Make I still get these errors

argument "begin" of operator "slice" has unknown type "tuple of <>, required"
argument "end" of operator "slice" has unknown type "tuple of <>, required"
argument "step" of operator "slice" has unknown type "tuple of <>, optional, default=[]"
argument "end" of operator "slice_axis" has unknown type ", required"

Could you post your make log here?

Here is a link to the make clean

looks like your cpp package failed. Do you use cpp as front end instead of Python? Your libmxnet seems ok.

I would obviously like to use the cpp frontend at some point. I had a look into the OpWrapperGenerator.py script and the issue seems to be that the MXSymbolGetAtomicSymbolInfo function (called from the OpWrapperGenerator.py script in the ParseAllOps function) returns a blank string for the types of the begin, end, and step parameters. Why would this be failing?

Which version of MXNet were you building from the source? the cpp-package/include/mxnet-cpp/op.h is not there in the master and 1.6 branch?

I have been trying to build the master branch

That’s interesting. Could you double check if you are working on a clean branch?

I just purged my folder, pulled, and built everything again. Still getting the same issue. Here is the new make log and also make/config.mk. This is the commit that I am currently building in master.

I could not repro the error that you met. What does your ‘git status’ show in your /media/RAID/Projects/mxnet say?

And could you also show “git remote -v” and “git submodule” result in the same folder?

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   Makefile
	modified:   make/config.mk

Apparently my system has a different installation configuration for opencv, so I needed to tweak the opencv include path

diff --git a/Makefile b/Makefile
index 49ba8fe00..88b3d2dd1 100644
--- a/Makefile
+++ b/Makefile
@@ -161,7 +161,7 @@ endif
 ifeq ($(USE_OPENCV), 1)
        CFLAGS += -DMXNET_USE_OPENCV=1
        ifneq ($(filter-out NONE, $(USE_OPENCV_INC_PATH)),)
-               CFLAGS += -I$(USE_OPENCV_INC_PATH)/include
+               CFLAGS += -I$(USE_OPENCV_INC_PATH)
                ifeq ($(filter-out NONE, $(USE_OPENCV_LIB_PATH)),)
 $(error Please add the path of OpenCV shared library path into `USE_OPENCV_LIB_PATH`, when `USE_OPENCV_INC_PATH` is not NONE)
                endif


$ git remote -v
origin	git@github.com:apache/incubator-mxnet.git (fetch)
origin	git@github.com:apache/incubator-mxnet.git (push)


$ git submodule
 3efc489b55385936531a06ff83425b719387ec63 3rdparty/dlpack (v0.2-9-g3efc489)
 c8f7f9c81dfcf2489d3bbc6d3b2a9200a20fbcde 3rdparty/dmlc-core (v0.3-136-gc8f7f9c)
 eb9225ce361affe561592e0912320b9db84985d0 3rdparty/googletest (release-1.8.0-1680-geb9225ce)
 8e96ef49488c65e0738c552cec5c0563ab92c1af 3rdparty/mkldnn (v0.1-rc-3907-g8e96ef494)
 c3cceac115c072fb63df1836ff46d8c60d9eb304 3rdparty/nvidia_cub (v1.8.0)
 f4745fcaff868a519834917c657f105a8eef2f53 3rdparty/onnx-tensorrt (release/6.0~10)
 b76842ed16984ae5edcbbc4b00a94fda20419431 3rdparty/openmp (remotes/origin/release_90)
 34fd45cae457d59850fdcb2066467778d0673f21 3rdparty/ps-lite (v1-208-g34fd45c)
 9bd2c7b44208ed992061f8c2688e1137357f1db1 3rdparty/tvm (v0.4-2109-g9bd2c7b4)

Do you copy out make/config.mk to mxnet root foler (same as Makefile) or just leave it there? And you still find op.h file under cpp-package/include/mxnet-cpp folder? I am curious what “ls” look like in that folder.

I can repro you error now. Anyway, libmxnet is OK. Please file a issue on the mxnet on github for the cpp-package.

I am leaving config.mk inside of the make folder.

$ ls cpp-package/include/mxnet-cpp
base.h       executor.h     io.h       kvstore.hpp     model.h      MxNetCpp.h   operator.h    op_map.h     optimizer.hpp  symbol.h
contrib.h    executor.hpp   io.hpp     lr_scheduler.h  monitor.h    ndarray.h    operator.hpp  op_suppl.h   op_util.h      symbol.hpp
CPPLINT.cfg  initializer.h  kvstore.h  metric.h        monitor.hpp  ndarray.hpp  op.h          optimizer.h  shape.h

I will file an issue shortly

Try a dirty workaround in the OpWrapperGenerator.py, if you want to give it a try. But I have not tested with real cpp examples.

I add two type at the bottom of the typeDict:

typeDict = {'boolean':'bool',\
        'boolean or None':'dmlc::optional<bool>',\
        'Shape(tuple)':'Shape',\
        'Symbol':'Symbol',\
        'NDArray':'Symbol',\
        'NDArray-or-Symbol':'Symbol',\
        'Symbol[]':'const std::vector<Symbol>&',\
        'Symbol or Symbol[]':'const std::vector<Symbol>&',\
        'NDArray[]':'const std::vector<Symbol>&',\
        'caffe-layer-parameter':'::caffe::LayerParameter',\
        'NDArray-or-Symbol[]':'const std::vector<Symbol>&',\
        'float':'mx_float',\
        'real_t':'mx_float',\
        'int':'int',\
        'int (non-negative)': 'uint32_t',\
        'long (non-negative)': 'uint64_t',\
        'int or None':'dmlc::optional<int>',\
        'float or None':'dmlc::optional<float>',\
        'long':'int64_t',\
        'double':'double',\
        'double or None':'dmlc::optional<double>',\
        'Shape or None':'dmlc::optional<Shape>',\
        'string':'const std::string&',\
        'tuple of <float>':'nnvm::Tuple<mx_float>', \
        'tuple of <>':'mxnet::cpp::Shape', \
        '': 'index_t' }

Especially the last one, it does not make any sense. Looks like the get type info has problem.

That workaround does allow it succeed

See if you can run any cpp example?

Do you know of an example that uses the slice op? Or is that not important for this test?

Issue has been created here

1 Like