I am using the following code to use tensorrt to speed up inference but got some errors:
sym, arg_params, aux_params = mx.model.load_checkpoint(prefix, epoch)
trt_sym = sym.get_backend_symbol(‘TensorRT’)
mx.contrib.tensorrt.init_tensorrt_params(trt_sym, arg_params, aux_params)
mx.contrib.tensorrt.set_use_fp16(False)
self.model = trt_sym.simple_bind(ctx=self.ctx, data = (1,3,image_size[0], image_size[1]), grad_req=‘null’, force_rebind=True)
self.model.copy_params_from(arg_params, aux_params)
File “/home/qiuzhewei/RetinaFace/retinaface.py”, line 224, in init
trt_sym = sym.get_backend_symbol(‘TensorRT’)
File “/opt/conda/lib/python3.7/site-packages/mxnet/symbol/symbol.py”, line 2564, in get_backend_symbol
check_call(_LIB.MXGenBackendSubgraph(self.handle, c_str(backend), ctypes.byref(out)))
File “/opt/conda/lib/python3.7/site-packages/mxnet/base.py”, line 253, in check_call
raise MXNetError(py_str(LIB.MXGetLastError()))
mxnet.base.MXNetError: [09:39:00] src/c_api/…/operator/subgraph/subgraph_property.h:367: Check failed: it != prop_ptr_map.end(): SubgraphProperty TensorRT is not found in SubgraphPropertyRegistry
Stack trace:
[bt] (0) /opt/conda/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x4a357b) [0x7f683cbe657b]
[bt] (1) /opt/conda/lib/python3.7/site-packages/mxnet/libmxnet.so(MXGenBackendSubgraph+0x1ab) [0x7f683ed5a5db]
[bt] (2) /opt/conda/lib/python3.7/lib-dynload/…/…/libffi.so.6(ffi_call_unix64+0x4c) [0x7f6876a4aec0]
[bt] (3) /opt/conda/lib/python3.7/lib-dynload/…/…/libffi.so.6(ffi_call+0x22d) [0x7f6876a4a87d]
[bt] (4) /opt/conda/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce) [0x7f6876ec9f7e]
[bt] (5) /opt/conda/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(+0x139b4) [0x7f6876eca9b4]
[bt] (6) python(_PyObject_FastCallKeywords+0x49b) [0x55b2e1d13d2b]
[bt] (7) python(_PyEval_EvalFrameDefault+0x537e) [0x55b2e1d6f7ae]
[bt] (8) python(_PyFunction_FastCallKeywords+0xfb) [0x55b2e1d1279b]
Anyone has some idea what happens?
Any help will be appreciate!