HybridBlock.hybrid_forward: Extra arguments for parameters


#1

Hello, I am referring to the example in
https://mxnet.incubator.apache.org/tutorials/gluon/hybrid.html

While I appreciate the simplication coming from the extra fc3_weight argument in hybrid_forward, I don’t really understand how this works in general.

For example, how is this argument named? In your example, the member self.fc3_weight and the internal name ‘fc3_weight’ are the same. But what if this is not the case? Say, the internal name is passed in as a string …

Also, in your example, there is only a single parameter fc3_weight. What if there are two or more? In which order do they appear in the signature?

I am quite confused and hope this is documented somewhere. It does not seem obvious.


#2

Hi,

The example is just a template/guideline. You don’t have to stick to the method signature provided and you can name the argument anything not just fc3_weight. You will have to stick with the method signature you defined in your hybrid_forward implementation when you do a forward pass on the network though. Specifically, if there are one or two extra parameters you need to pass, you can pass them in any order in your hybrid_forward implementation after the F and x argument but you just need to preserve that order when you do a forward pass on the network using net()

Hopefully that clarifies things. Feel free to ask any other follow-up questions you might have


#3

Sorry for not being clear, but that is not what I am asking. The extra argument fc3_weight is not part of the signature as one calls it. It is appended automatically. The idea is that the gluon parameter fc3_weight is unwrapped into a symbol or ndarray, depending on what F is.

My question is: if I have two gluon parameters as members, or more, then how do I name these extra arguments?


#4

Hi, sorry for the misunderstanding. The internal name doesn’t need to match the member name and the order of the arguments doesn’t matter. See below a modification of the example that illustrates this:

import mxnet as mx
from mxnet import gluon
from mxnet.gluon import nn

mx.random.seed(42)

class Net(gluon.HybridBlock):
    def __init__(self, **kwargs):
        super(Net, self).__init__(**kwargs)
        with self.name_scope():
            # layers created in name_scope will inherit name space
            # from parent layer.
            self.conv1 = nn.Conv2D(6, kernel_size=5)
            self.pool1 = nn.MaxPool2D(pool_size=2)
            self.conv2 = nn.Conv2D(16, kernel_size=5)
            self.pool2 = nn.MaxPool2D(pool_size=2)
            self.fc1 = nn.Dense(120)
            self.fc2 = nn.Dense(84)
            # You can use a Dense layer for fc3 but we do dot product manually
            # here for illustration purposes.
            self.fc3_weight = self.params.get('fc3_weight', shape=(10, 84))
            self.fc4_w = self.params.get('fc4_weight', shape=(10, 1))

    def hybrid_forward(self, F, x, fc4_w, fc3_weight):
        # Here `F` can be either mx.nd or mx.sym, x is the input data,
        # and fc3_weight is either self.fc3_weight.data() or
        # self.fc3_weight.var() depending on whether x is Symbol or NDArray
        print(x)
        x = self.pool1(F.relu(self.conv1(x)))
        x = self.pool2(F.relu(self.conv2(x)))
        # 0 means copy over size from corresponding dimension.
        # -1 means infer size from the rest of dimensions.
        x = x.reshape((0, -1))
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = F.dot(x, fc3_weight, transpose_b=True)
        x = F.dot(x, fc4_w)
        return x

net = Net()
net.initialize()
x = mx.nd.random_normal(shape=(16, 1, 28, 28))
print net(x)
x = mx.nd.random_normal(shape=(16, 1, 28, 28))
print net(x)

#5

OK thanks,

so your example says that the argument name in hybrid_forward matches the member name, while the internal name does not matter.

This is also what I figured by playing around. Not knowing a lot about Python, I was just puzzled how this is done.

Bye, Matthias