HybridBlock Incompatible attr in node error


#1

My HybridBlock code runs fine when net.hybridize() is commented out, but, I get the following error when it is not commented out.

elemwise_op_common.h:122: Check failed: assign(&dattr, (*vec)[i]) Incompatible attr in node lyp_network2_mul12 at 1-th input: expected (32, 301, 512), got (32,1,1)

What are the possibilities that would cause that error?

Here is essentially the network code:

def doBatchDot(F, W, inVal):
    inVal1 = F.expand_dims(inVal, axis=2)
    temp1 = F.batch_dot(W, inVal1)
    temp2 = F.flatten(temp1)
    return temp2

 def hybrid_forward(self, F, x, i1, i2, i3, i4, i5, i6):
       ...blah...
       H1 = self.activation(doBatchDot(F, W0, self.dropout0(x)) + b0)
       H2 = self.activation(doBatchDot(F, W1, self.dropout1(H1)) + b1)
       H3 = doBatchDot(F, W2, self.dropout2(H2)) + b2
       return H3

Thanks!


#2

what is self.activation? If it is self-defined, it should be a HybridBlock.

Suppose it is relu, we can do F.relu(doBatchDot(F, W0, self.dropout0(x)) + b0)


#3

It seems the assign (=) has mismatched LHS and RHS shapes.


#4

I have indeed made self.activation to be a HybridBlock as follows. So, that is not the reason; something else must be the reason for the error.

class MyELUActivationBlock(gluon.HybridBlock):

    def hybrid_forward(self, F, x):
            return F.LeakyReLU(x, slope=self.negativeSideMultiplier)

self.activation = MyELUActivationBlock(1.0)

There is no obvious mismatch between the LHS and RHS shapes. Otherwise, the code wouldn’t have worked with hybridize() commented out; right?

By the way, searching on the internet, it does look like others have run into this error in the past as well, though not in the context of HybridBlocks.

https://github.com/apache/incubator-mxnet/issues/6213

https://github.com/apache/incubator-mxnet/issues/5420


#5

For the benefit anyone else who may run into this in future:

Using F.broadcast_mul instead of * to do the multiplication made the error go away.


#6

Yes, this is documented in the symbol API that implicit broadcasting is not supported.

Basically, in your hybrid_forward method you either want to use parts of the API that are the same between the ndarray and symbol modules or conditionalize based on whether the block is active (or the value of F).

Another thing to avoid is the use of the [] slicing notation.