How to train specific layers using gluon with different learning rate?


It’s actually 2 questions.

  1. For example, I want to train the frst conv layer while the other layers are freezed. However, it seems that I can only keep the base layer unchanged while training the last layer:

for X, y in train_data:
y = y.astype(‘float32’).as_in_context(ctx)
output_features = net.features(X.as_in_context(ctx))
with autograd.record():
outputs = net.output(output_features)
l = loss(outputs, y)
train_l += l.mean().asscalar()

  1. Is there any way to train different block of a net with different learning rate? Fro example , the lr of the output part is still lr, while the middle layers of the model are trained with lr/3, and the base is trained with even smaller lr/10. I know that in it’s very easy to do so:


How can I do above tasks?
Thanks a lot!