I would like to build a network using a symbol and parameter class for inference but use a custom hybrid_forward to be able to save gradients of one hidden layer wrt some other hidden layer. Is there a way to read symbol file and parse its symbols in order to create a network bu calling nn.HybridSequential.add later? Please note that I know how to use import to load the model and run inference on it and I understand that I can use autograd to get the gradients of the hidden layers wrt to input; however, my specific task requires custom hybrid_forward. I am thinking of creating a new network with the same layers as the loaded network and loading the parameter while using a custom hybrid_forward.
hi,
yes your idea would be the way to go for this. if your symbol model is not too complicated you can create a custom network that extends gluon.HybridBlock and in your __init__ function for your network, create the layers as in the symbolic model. then implement the hybrid_forward the way you need to compute. you can then initialize the parameters of your custom network with the saved parameters from your symbolic model and then you should be good to go.
if you don’t know the network architecture of you symbolic network, you can read in a custom model created by symbol into gluon via gluon.nn.SymbolBlock but you won’t be able to change the forward function. you can print the network however to see the architecture and then write your custom block based on that.
Thanks Sad, I am trying to have a general solution which can serve all different models so I can only create custom block dynamically and after reading symbol file which I do not know how to do it. I cannot pre-define the network. Is there a way to read the symbol file in the code and then create the block based on that?