Symbolic mode: how to block the gradient in the graph?

MXNet Model Server
1

I’m training a image classification model using the mxnet.sym api.

A brief illustration of my model is as below:

%E5%9B%BE%E7%89%871
%E5%9B%BE%E7%89%8711255×315 8.95 KB

Besides the prediction and label, the loss function accepts a weights from the intermediate features
as input.

However, I don’t want the gradient to propagate through the weights variable.
In other words, I need the weights to acting as a static ndarray (just like the Lable),
rather than a Variable.

So how can I disable the gradient back-prop through the variable weights?

Currently I use the mx.symbol.BlockGrad():

feature = conv(data)
weights = linear1(feature)
weights = mx.symbol.BlockGrad(weights)
prediction = linear2(feature)
loss = softmax_loss(prediction, target)

Is it correct to do so?

2

Hi @kaizhao,

Yes that’s the way to do it.

After doing weights = mx.symbol.BlockGrad(weights), weights won’t be contributing to the gradients the nodes before linear1(feature) (this node included).

1 Like
3

Hi @spanev, thanks for solving my problem!

1 Like