How to fuse Batchnorm into convolution to speedup inference?

1

I found that the random initialize BN and using real weight initialized BN can have different computation time, and the later one cost much more. So I wonder if there is a way to fuse BN to convolution in MXNet ?

Thanks in advance.

2

Hi @TriLoon,

You might want to check out the TVM project for this. Operator fusion is one of its key features.

3

Thanks, I will check it out.

4

A very easy implementation, use get_backend_symbol() to fuse ops. But I am not sure whether should I do some changes to the parameters. Now I only use get_backend_symbol to change the backend of symbol network, and I noticed that the symbol name (obtained by get_internals()) has changed, but I do nothing to corresponding parameters (the key name of params not change). The latency has been reduced.