Different ImageNet accuracy between caffe model and caffe-converted MXNet model


#1

Hi, I tested caffe-converted ImageNet pretrain model and found inferior performance of MXNet model.

I converted Kaiming’s ResNet 50 and 101 from caffe model to mxnet model. For ResNet 50, the caffe model gives accuracy of 75.3 but the converted mxnet model gives accuracy of 74.4. Is this normal? I followed the instructions in mxnet image-classification example to generate recordIO file with JPEG quality=95 on the validation set.


#2

Conversions often cause acc drop due to subtle differences in numeric operations.
You can use mxnet model zoo to get native mxnet models. It does include resnet101


#3

Thanks for your reply. However, I would like to use resnet-v1 (post activation). I found the models in (http://data.mxnet.io/models/imagenet/resnet/) are resnet-v2 (pre activation). Is there any way to get resnet-v1 pretrained model?


#4

Gluon has it.

https://mxnet.incubator.apache.org/api/python/gluon/model_zoo.html?highlight=resnet#mxnet.gluon.model_zoo.vision.resnet101_v1


#5

If you still want to use symbol+module, you can use gluon to get the model and then use export to save it to module checkpoint. Then load it back with module.

model = mxnet.gluon.model_zoo.vision.resnet101_v1(pretrained=True)
model.export('save')
module = mx.mod.Module.load('save')