It’s actually 2 questions.
- For example, I want to train the frst conv layer while the other layers are freezed. However, it seems that I can only keep the base layer unchanged while training the last layer:
for X, y in train_data:
y = y.astype(‘float32’).as_in_context(ctx)
output_features = net.features(X.as_in_context(ctx))
outputs = net.output(output_features)
l = loss(outputs, y)
train_l += l.mean().asscalar()
Is there any way to train different block of a net with different learning rate? Fro example , the lr of the output part is still lr, while the middle layers of the model are trained with lr/3, and the base is trained with even smaller lr/10. I know that in fast.ai it’s very easy to do so:
How can I do above tasks?
Thanks a lot!