Consult the hybrid_forward interface of the _Conv class


#1

Why does the hybrid_forward function of the _Conv class have the arguments weight and bias , but don’t use self.weight in the constructor? How is the parameter self.weight updated?

def hybrid_forward(self, F, x, weight, bias=None):
        if bias is None:
            act = getattr(F, self._op_name)(x, weight, name='fwd', **self._kwargs)
        else:
            act = getattr(F, self._op_name)(x, weight, bias, name='fwd', **self._kwargs)
        if self.act is not None:
            act = self.act(act)
        return act

#2

When you’re using a hybrid block, F is ndarray when not hybridized, but it is symbol when block is hybridized. When block is hybridized, during hybrid_forward call you’re not actually performing the computation as you would in imperative mode, but rather you’re constructing the symbolic graph of the computation. When constructing a symbolic graph, parameters of the network have to be abstracted away and represented by a symbol. Once the entire computational graph is composed, then for evaluating the graph, these symbols are bound to the actual parameter values.

Under the hood, gluon.HybridBlock takes care of creating symbols for each parameter and when gluon.HybridBlock.forward() is called, it constructs a symbol for each parameter of the network and passes it down to the hybrid_forward() call which is implemented by the child class (for example the _Conv class in your question).

When block is not hybridized, however, gluon.HybridBlock simply passes, for example self.weight.get(ctx), as weight to hybrid_forward().

This pattern allows the child block to define its parameters and then expect all those parameters to be automatically passed in hybrid_forward() call. When running in imperative mode, these parameters are actual NDArray objects and when in hybridized mode, these are symbols representing the NDArray objects. In either case, you can simply just use what’s passed into the hybrid_forward() call and assume that everything will just magically work :slight_smile: