How to convert a CNN from keras to mxnet?

Gluon
1

Hi,

this theme is originally from stackoverflow. Unfortunately, the restrictions there don’t allow me to ask advanced questions, so Foivos told me that it would be a good idea to continue the conversation here.

In summary, it was about a keras net, which I wanted to convert to gluon, but which turned out to be difficult. Foivos then sent me the (probably) exact translation of the network. Unfortunately I couldn’t find out if the networks behave the same, because the results of the Class Activation Mapping (CAM), which I also converted from keras to gluon, were different (for details see: https://stackoverflow.com/questions/55186629/how-to-convert-a-cnn-from-keras-to-mxnet/55604738?noredirect=1#comment98154231_55604738).

Now I would like to continue with the topic where I had to stop at stackoverflow.

To be honest, I’m not quite sure if the net is doing the same thing. After the training I created a Class Activation Mapping (CAM) for both keras and mxnet to show me the relevant regions over Europe. Unfortunately I get different results. But maybe my implementation is wrong?

keras:

def get_cam(airport):
    fname = os.path.normpath( "model_{}.h5".format(airport) )
    model = load_model(fname)

    dry_weights  = model.layers[-1].get_weights()[0][:, 0].reshape((20,30,256))
    rain_weights = model.layers[-1].get_weights()[0][:, 1].reshape((20,30,256))
    get_output   = K.function([model.layers[0].input], [model.layers[-3].output, model.layers[-1].output])

    cam = np.zeros((20,30), dtype=np.float32)
    for depth in range(256):
        for i in range(30):
            for j in range(20):
                cam[j, i] += rain_weights[j, i, depth]

    cam /= np.max(cam)
    im = Image.fromarray(np.uint8(cm.jet(cam)*255))
    im = im.resize((120,80), Image.ANTIALIAS)
    #heatmap = cv2.applyColorMap(np.uint8(255*cam), cv2.COLORMAP_JET)
    im.save("{}_rain.png".format(airport))

mxnet:

def get_cam(model):
    dry_weights  = model.last_layer.weight.data()[0].reshape((256,20,30)).asnumpy()
    rain_weights = model.last_layer.weight.data()[1].reshape((256,20,30)).asnumpy()

    cam = np.zeros((20,30), dtype=np.float32)
    for depth in range(256):
        for i in range(30):
            for j in range(20):
                cam[j, i] += rain_weights[j, i, depth]

    cam /= np.max(cam)
    im = Image.fromarray(np.uint8(cm.jet(cam)*255))
    im = im.resize((120,80), Image.ANTIALIAS)
    #heatmap = cv2.applyColorMap(np.uint8(255*cam), cv2.COLORMAP_JET)
    im.save("{}_rain.png".format(airport))

call it with:

model = YourNet()
model.load_parameters(in_file, ctx=ctx)
get_cam(model)

I thought, since the channels appear first, I would also have to change the order of the reshape. In any case, I also tried the original order, but with equally bad results. Well, that’s what I get in both cases:

comparison
comparison.PNG500×747 142 KB

Any ideas?

2

Hi Stefan,

welcome to the forum. Can you please provide a reference implementation for Keras (somewhere on the github + paper) on which you are based? I’ve found several things online for CAM, and am not really sure what you are doing.

As a side note, the code for mxnet has at least a bug, rain_weights[j, i, depth], here depth should be in the first slot rain_weights[depth,j, i] but I suspect this is a syntactic error, as the code breaks if you use wrong indexing.

3

Hi Feevos,

of course! Maybe that would have been the best course of action from the beginning… I’m sorry for the delay.

github: https://github.com/prl900/DeepWeather
paper: https://github.com/prl900/DeepWeather/blob/master/1stDeepStructWS_paper_2.pdf

Regards,
Stefan

1 Like