While I appreciate the simplication coming from the extra fc3_weight argument in hybrid_forward, I don’t really understand how this works in general.
For example, how is this argument named? In your example, the member self.fc3_weight and the internal name ‘fc3_weight’ are the same. But what if this is not the case? Say, the internal name is passed in as a string …
Also, in your example, there is only a single parameter fc3_weight. What if there are two or more? In which order do they appear in the signature?
I am quite confused and hope this is documented somewhere. It does not seem obvious.
The example is just a template/guideline. You don’t have to stick to the method signature provided and you can name the argument anything not just fc3_weight. You will have to stick with the method signature you defined in your hybrid_forward implementation when you do a forward pass on the network though. Specifically, if there are one or two extra parameters you need to pass, you can pass them in any order in your hybrid_forward implementation after the F and x argument but you just need to preserve that order when you do a forward pass on the network using net()
Hopefully that clarifies things. Feel free to ask any other follow-up questions you might have
Sorry for not being clear, but that is not what I am asking. The extra argument fc3_weight is not part of the signature as one calls it. It is appended automatically. The idea is that the gluon parameter fc3_weight is unwrapped into a symbol or ndarray, depending on what F is.
My question is: if I have two gluon parameters as members, or more, then how do I name these extra arguments?
Hi, sorry for the misunderstanding. The internal name doesn’t need to match the member name and the order of the arguments doesn’t matter. See below a modification of the example that illustrates this:
import mxnet as mx
from mxnet import gluon
from mxnet.gluon import nn
mx.random.seed(42)
class Net(gluon.HybridBlock):
def __init__(self, **kwargs):
super(Net, self).__init__(**kwargs)
with self.name_scope():
# layers created in name_scope will inherit name space
# from parent layer.
self.conv1 = nn.Conv2D(6, kernel_size=5)
self.pool1 = nn.MaxPool2D(pool_size=2)
self.conv2 = nn.Conv2D(16, kernel_size=5)
self.pool2 = nn.MaxPool2D(pool_size=2)
self.fc1 = nn.Dense(120)
self.fc2 = nn.Dense(84)
# You can use a Dense layer for fc3 but we do dot product manually
# here for illustration purposes.
self.fc3_weight = self.params.get('fc3_weight', shape=(10, 84))
self.fc4_w = self.params.get('fc4_weight', shape=(10, 1))
def hybrid_forward(self, F, x, fc4_w, fc3_weight):
# Here `F` can be either mx.nd or mx.sym, x is the input data,
# and fc3_weight is either self.fc3_weight.data() or
# self.fc3_weight.var() depending on whether x is Symbol or NDArray
print(x)
x = self.pool1(F.relu(self.conv1(x)))
x = self.pool2(F.relu(self.conv2(x)))
# 0 means copy over size from corresponding dimension.
# -1 means infer size from the rest of dimensions.
x = x.reshape((0, -1))
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = F.dot(x, fc3_weight, transpose_b=True)
x = F.dot(x, fc4_w)
return x
net = Net()
net.initialize()
x = mx.nd.random_normal(shape=(16, 1, 28, 28))
print net(x)
x = mx.nd.random_normal(shape=(16, 1, 28, 28))
print net(x)