MXNet to ONNX to TensorRT, how to interpret trt output to match mxnet output

1

Hello,
I’ve a code in MXNET which I exported to ONNX, then from ONNX imported to TensorRT.

I’m using onnx-tensorrt(GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX) in order to run the inference.

I’ve got an output after using

  1. trt_outputs = common.do_inference(context, bindings=bindings, inputs=inputs, outputs=outputs, stream=stream)

Also I’ve got an output when I do a forward pass in my MXNET (on that output I find the bboxes values for the face).

Question: How can I convert the TensorRT’s inference output to match the MXNET’s inference output so I can classify the faces with the bboxes?

Also, maybe I don’t look at the right place and I need to ignore MXNET’s output and interpret ONNX’s output and use that instead? (I also verified that ONNX has the same output)

2

I am not sure why the output is different, but if you use MXNet-TensorRT binding, you should receive the same output. This approach allows you to get inference performance improvement, but without the need to manually export the model outside of MXNet.

Take a look into this explanation https://mxnet.incubator.apache.org/versions/master/tutorials/tensorrt/inference_with_trt.html

3

Hi Sergey,
the output shouldn’t be different, I mean I guess the array is flattened or need to be reshaped somehow to match the original mxnet/onnx output. I’ll debug it later today and update you, I just got my PyCharm Pro integerated with the nvidia-docker so I can debug now in PyCharm instead of using print methods.

Also, I’m trying to find a project like the yolov3 of NVIDIA’s TensorRT and see the onnx output and the tensorrt output, but as far as I can see the samples folder just comes with the onnx file, without any inference code for the yolov3 onnx, just for the tensorrt, meaning that I can’t get the output of the onnx file - in order to compare them.

I’ll also look at nvidia’s samples (for other onnx-trt conversion projects) too if available, I’ll update asap. (I’ll also check the MXNet-TRT binding if I won’t figure it out until then)
If you have any ideas until then, I’d love to know.