Convert_model to float16

Following the recent changes in the amp package I tried to convert a trained mxnet model from float32 to float16 using the convert_model script from https://github.com/apache/incubator-mxnet/pull/15118

However, I got an error that the data type is unsupported:

Error in operator retinanet0_multiboxprior0: [16:53:35] /home/local/ANT/dfferstl/software/incubator-mxnet/include/mxnet/operator.h:228: Check failed: in_type->at(i) == mshadow::default_type_flag || in_type->at(i) == -1: Unsupported data type 2

or if I try to use the implicit cast from [solved] Network in float16 I get the same error at the batchnorm gamma:

MXNetError: Error in operator retinanet0_batchnorm0_fwd: [16:52:35] /home/local/ANT/dfferstl/software/incubator-mxnet/src/operator/nn/batch_norm.cc:370: Check failed: (*in_type)[i] == dtype_param (2 vs. 0) : This layer requires uniform type. Expected 'float32' v.s. given 'float16' at 'gamma'

Is this simply not yet supported or is there any way to solve this?

Thanks,
David

I think it is an issue with the multibox prior operator that doesn’t support the float16 mode.
Have you tried using the gluonCV target generator instead?
For example: https://github.com/dmlc/gluon-cv/blob/master/gluoncv/model_zoo/ssd/target.py

Alternatively, you can also cast selectively parameters by doing:

net.collect_params('.*gamma.*').cast(...)

Looks like the legacy op infer type doesnt support any dtype but FP32. Until this is fixed, the default AMP lists should be tweaked. Can you try moving this to FP32_FUNCS list from WIDEST_TYPE_CASTS list to see if this solves your issue.

EDIT: : Forgot to put the link earlier: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/contrib/amp/lists/symbol.py#L579 move this to FP32_FUNCS list in the same module. Would you also be able to provide the link to the model you are trying.

Unfortunately we are using the MultiBoxTarget/MultiBoxPrior/MultiBoxDetection stuff from the contrib library. Does this mean that this won’t work with 16Bit?

More generally, how should a custom HybridBlock / HybridSequential look like in order to be castable?

I have opened a PR to move the unsupported ops to FP32 lists for now: https://github.com/apache/incubator-mxnet/pull/16668. This should fix your issue with inference with the converted model now. (Ideally, we should support FP16 for all these three ops which is a TODO: https://github.com/apache/incubator-mxnet/issues/16669)