I want to use the mxnet.contrib.svrg_optimization module with gluon.trainer but I’m having some issues with it. I’ve tried to use a simple softmax layer like
You get this exception, because SVRGModule is working only with Module API.
What you can do is to write your model in Gluon, export to .json file and later load it back as a Symbol. You also would need to add your expected output to the loaded back model, because original Gluon one wouldn’t have a label associated with that. Take a look into this example:
import mxnet as mx
from mxnet.contrib.svrg_optimization.svrg_module import SVRGModule
from mxnet.gluon import nn
from mxnet.initializer import Normal
ctx = mx.cpu()
net = nn.HybridSequential()
net.add(nn.Dense(10))
net.initialize(Normal(sigma=0.01))
# hybridization and forward pass needed to make model exportable
net.hybridize(static_alloc=True)
net(mx.random.uniform(shape=(10, 5)))
# this would save model to json file
net.export("net")
# We load model back and attach SoftmaxOutput
# Without attaching softmax SVRGModule would complain that it cannot find the label
deserialized_net = mx.symbol.load("net-symbol.json")
net_with_softmax_output = mx.symbol.SoftmaxOutput(data=deserialized_net)
svrg = SVRGModule(net_with_softmax_output,
data_names=['data'],
label_names=['softmaxoutput0_label'],
update_freq=2)
I haven’t tried to train the module, because I don’t have training data. But feel free to try, and let me know if it works.