Strange compile errors

I would obviously like to use the cpp frontend at some point. I had a look into the OpWrapperGenerator.py script and the issue seems to be that the MXSymbolGetAtomicSymbolInfo function (called from the OpWrapperGenerator.py script in the ParseAllOps function) returns a blank string for the types of the begin, end, and step parameters. Why would this be failing?

Which version of MXNet were you building from the source? the cpp-package/include/mxnet-cpp/op.h is not there in the master and 1.6 branch?

I have been trying to build the master branch

That’s interesting. Could you double check if you are working on a clean branch?

I just purged my folder, pulled, and built everything again. Still getting the same issue. Here is the new make log and also make/config.mk. This is the commit that I am currently building in master.

I could not repro the error that you met. What does your ‘git status’ show in your /media/RAID/Projects/mxnet say?

And could you also show “git remote -v” and “git submodule” result in the same folder?

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   Makefile
	modified:   make/config.mk

Apparently my system has a different installation configuration for opencv, so I needed to tweak the opencv include path

diff --git a/Makefile b/Makefile
index 49ba8fe00..88b3d2dd1 100644
--- a/Makefile
+++ b/Makefile
@@ -161,7 +161,7 @@ endif
 ifeq ($(USE_OPENCV), 1)
        CFLAGS += -DMXNET_USE_OPENCV=1
        ifneq ($(filter-out NONE, $(USE_OPENCV_INC_PATH)),)
-               CFLAGS += -I$(USE_OPENCV_INC_PATH)/include
+               CFLAGS += -I$(USE_OPENCV_INC_PATH)
                ifeq ($(filter-out NONE, $(USE_OPENCV_LIB_PATH)),)
 $(error Please add the path of OpenCV shared library path into `USE_OPENCV_LIB_PATH`, when `USE_OPENCV_INC_PATH` is not NONE)
                endif


$ git remote -v
origin	git@github.com:apache/incubator-mxnet.git (fetch)
origin	git@github.com:apache/incubator-mxnet.git (push)


$ git submodule
 3efc489b55385936531a06ff83425b719387ec63 3rdparty/dlpack (v0.2-9-g3efc489)
 c8f7f9c81dfcf2489d3bbc6d3b2a9200a20fbcde 3rdparty/dmlc-core (v0.3-136-gc8f7f9c)
 eb9225ce361affe561592e0912320b9db84985d0 3rdparty/googletest (release-1.8.0-1680-geb9225ce)
 8e96ef49488c65e0738c552cec5c0563ab92c1af 3rdparty/mkldnn (v0.1-rc-3907-g8e96ef494)
 c3cceac115c072fb63df1836ff46d8c60d9eb304 3rdparty/nvidia_cub (v1.8.0)
 f4745fcaff868a519834917c657f105a8eef2f53 3rdparty/onnx-tensorrt (release/6.0~10)
 b76842ed16984ae5edcbbc4b00a94fda20419431 3rdparty/openmp (remotes/origin/release_90)
 34fd45cae457d59850fdcb2066467778d0673f21 3rdparty/ps-lite (v1-208-g34fd45c)
 9bd2c7b44208ed992061f8c2688e1137357f1db1 3rdparty/tvm (v0.4-2109-g9bd2c7b4)

Do you copy out make/config.mk to mxnet root foler (same as Makefile) or just leave it there? And you still find op.h file under cpp-package/include/mxnet-cpp folder? I am curious what “ls” look like in that folder.

I can repro you error now. Anyway, libmxnet is OK. Please file a issue on the mxnet on github for the cpp-package.

I am leaving config.mk inside of the make folder.

$ ls cpp-package/include/mxnet-cpp
base.h       executor.h     io.h       kvstore.hpp     model.h      MxNetCpp.h   operator.h    op_map.h     optimizer.hpp  symbol.h
contrib.h    executor.hpp   io.hpp     lr_scheduler.h  monitor.h    ndarray.h    operator.hpp  op_suppl.h   op_util.h      symbol.hpp
CPPLINT.cfg  initializer.h  kvstore.h  metric.h        monitor.hpp  ndarray.hpp  op.h          optimizer.h  shape.h

I will file an issue shortly

Try a dirty workaround in the OpWrapperGenerator.py, if you want to give it a try. But I have not tested with real cpp examples.

I add two type at the bottom of the typeDict:

typeDict = {'boolean':'bool',\
        'boolean or None':'dmlc::optional<bool>',\
        'Shape(tuple)':'Shape',\
        'Symbol':'Symbol',\
        'NDArray':'Symbol',\
        'NDArray-or-Symbol':'Symbol',\
        'Symbol[]':'const std::vector<Symbol>&',\
        'Symbol or Symbol[]':'const std::vector<Symbol>&',\
        'NDArray[]':'const std::vector<Symbol>&',\
        'caffe-layer-parameter':'::caffe::LayerParameter',\
        'NDArray-or-Symbol[]':'const std::vector<Symbol>&',\
        'float':'mx_float',\
        'real_t':'mx_float',\
        'int':'int',\
        'int (non-negative)': 'uint32_t',\
        'long (non-negative)': 'uint64_t',\
        'int or None':'dmlc::optional<int>',\
        'float or None':'dmlc::optional<float>',\
        'long':'int64_t',\
        'double':'double',\
        'double or None':'dmlc::optional<double>',\
        'Shape or None':'dmlc::optional<Shape>',\
        'string':'const std::string&',\
        'tuple of <float>':'nnvm::Tuple<mx_float>', \
        'tuple of <>':'mxnet::cpp::Shape', \
        '': 'index_t' }

Especially the last one, it does not make any sense. Looks like the get type info has problem.

That workaround does allow it succeed

See if you can run any cpp example?

Do you know of an example that uses the slice op? Or is that not important for this test?

Issue has been created here

1 Like

Slice is important in general. I did not check the tests.

Could this error also be related to the other issues we have been discussing, or is this something else?

$ cd python
$ pip install --user -e .
Obtaining file:///home/bidski/Projects/mxnet/python
    ERROR: Command errored out with exit status -6:
     command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/bidski/Projects/mxnet/python/setup.py'"'"'; __file__='"'"'/home/bidski/Projects/mxnet/python/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
         cwd: /home/bidski/Projects/mxnet/python/
    Complete output (1 lines):
    free(): invalid pointer
    ----------------------------------------
ERROR: Command errored out with exit status -6: python setup.py egg_info Check the logs for full command output.

I doubt it as python front end package is independent from cpp package.

Is it possible that the error is not in the cpp package, but in libmxnet?

same error here …

I downloaded the latest (commit head: 67b5d314a5a46bce4377668626) and build it with CMake 3.16.3, g++ 7.3.1, same error arose.

I changed the CMakeLists.txt (from commit de510582438ad5fad576eba1b) and now the code can be built correctly.