Update value of non-trainable parameters during training

1

Hi

I am using a sampling strategy during training that I think be implemented by changing a parameter of a HybridBlock during training. To give some context, I am implementing a graph convolutional neural network where I multiply an adjacency matrix A with a node features matrix X. I have the following block:

class GraphConv(HybridBlock):
    def __init__(self, A, in_units, out_units,
                 activation=lambda x: x, **kwargs):
        super().__init__(**kwargs)

        self.activation = activation

        self.in_units = in_units
        self.out_units = out_units

        self.A = self.params.get_constant('A', A)
        self.W = self.params.get(
            'W', shape=(self.in_units, self.out_units),
        )

    def hybrid_forward(self, F, X, A, W):
        aggregate = F.dot(A, X)
        propagate = self.activation(
            F.dot(aggregate, W))

        return propagate

I want to update the Constant A during each epoch during training, where I effectively subsample the full adjacency matrix. Any ideas on how to do this in the Gluon API?

2

You could use set_data to update A. For instance this code snippet updates A in each iteration, by multiplying it with a a constant.

A = np.ones((10,10))
net = GraphConv(A, 5, 2)
net.initialize()
net.hybridize()

for i in range(5):
    new_A = np.ones((10,10))*i
    net.A.set_data(new_A)
    print(net(mx.nd.ones((10,))))
1 Like