Loading from saved params - but "params not initialized" error

Gluon
1

Hey Everyone,

I just fine-tuned a pretrained gluon model (dense-net) and saved the params using net.collect_params().save('params_file_name").

I load the parameters using:

net = mx.gluon.model_zoo.vision.get_model("densenet121", classes=class_num, ctx=ctx)

net.load_params('./densenet121-tuned-2018-07-20-13:29:05.params', ctx=ctx, ignore_extra=True, allow_missing=True)

However, when I try to do any forward pass through the model, I get the following error:

RuntimeError: Parameter densenet4_conv0_weight has not been initialized. Note that you should initialize parameters and create Trainer with Block.collect_params() instead of Block.params because the later does not include Parameters of nested child Blocks

I’ve been looking across the forum and couldn’t find anything that helps, so figured I would post and get some help!

One thing to note, however, is that I can only load the params when i use ignore_extra=True AND allow_missing=True. When I leave both of those flags out, I get errors saying that parameters are missing in the params file. I looked at this post for a solution and tried setting prefix to “”, but it didn’t help: https://gist.github.com/aaronmarkham/1017664fe683596c614961112a867145

Not sure if the load params is the root cause of my issue, but has anyone else faced/overcome this issue?

Here is how I’m creating the densenet model prior to training:

densenet121_pretrained = vision.densenet121(pretrained=True)
dense_tune = vision.densenet121(prefix='', classes=class_num)
dense_tune.collect_params().initialize(ctx=ctx)
dense_tune.features = densenet121_pretrained.features

Thanks so much!

2

hi pn-train,

I wasn’t able to reproduce the issue you described exactly, but I got a different error when I tried to use net.collect_params().save('params_file_name'). I found out that if I used net.save_params() or net.save_parameters() I was able to save a finetuned model with a different number of classes, and reload the model and successfully do a forward pass.

Here’s the code I have that does this:

import mxnet as mx
import mxnet.gluon
import mxnet.ndarray as nd
from mxnet.gluon.model_zoo import vision

class_num = 2
file_name = "densenet_tune.params"

densenet121_pretrained = vision.densenet121(pretrained=True)

dense_tune = vision.densenet121(prefix='', classes=class_num)
dense_tune.collect_params().initialize(ctx=mx.cpu())
dense_tune.features = densenet121_pretrained.features
dense_tune(nd.ones((1, 3, 224, 224))) #forward pass to initialize parameters
dense_tune.save_parameters(file_name)

net = mx.gluon.model_zoo.vision.get_model("densenet121", classes=class_num, ctx=mx.cpu())
net.load_parameters(file_name)

print net(nd.ones((1, 3, 224, 224)))

If this doesn’t work for you, you can reply back to this post with your full code that shows the save, load and forward pass in order.

1 Like
3

Hi sad,

Thank you so much! This worked and you’re right, net.save_parameters() is much better. The original issue had to do with namescopes when loading the features.