MXNet Forum

Weird result when run with the autograd.record


#1

Hi!

I have some experiences with tensorflow and is new to mxnet.

I define a hybrid net which is composed of a 3D convolution net (SampleNet) and a pre-trained PSP net.

I use the following codes to fixed the parameters in the PSP net.

def net(input)
 trainer = gluon.Trainer(self.sample_net.collect_params(),'sgd',{'learning_rate':0.0001})
 with autograd.record():
        sample_result = SampleNet(input)
        predict_result = PSPNet(sample_result)

the problem is, when input an image (left, as the result of sample_result) when i output the predicted_result, i get very noisy output (Middle). and if i commented the autograd.record(), the predict_result seems decent (right). So what’s happened?

Any suggestion or debug direction would be appreciated.


#2

The command autograd.record puts your network in training mode, which means that layers such as dropout, BatchNorm will behave differently than in inference mode. By default, the output of a network is in autograd.predict_mode. Therefore when you comment out the line with autograd.record your network will use summary statistics (for BatchNorm) and average of the activations (dropout, instead of killing a random sample of them). Hope this helps.