Saving and loading bert model

[chidauri]

Hi, i have followed this https://gluon-nlp.mxnet.io/examples/sentence_embedding/bert.html to build a similar bert model,i want to know if there is a way to save this classifier and then use it in separate script to make predictions. Thanks.

[sad]

Is your separate script going to be in python still?

If so then you can get saved the parameters of the trained bert model using .save_parameters().

Recreate the model architecture in your inference script and the reload the save parameters using .load_parameters()

Essentially you want something like:

bert_arch, vocabulary = nlp.model.get_model('bert_12_768_12',
                                             dataset_name='book_corpus_wiki_en_uncased',
                                             pretrained=False, ctx=ctx, use_pooler=True,
                                             use_decoder=False, use_classifier=False)
bert_classifier = model.classification.BERTClassifier(bert_arch, num_classes=num_classes, dropout=dropout)
bert_classifier.load_parameters('bert_saved_parameters')

in your inference code.

[chidauri]

Thanks , i tried this and it worked

[TetianaGryshchuk]

@chidauri, please, explain how to predict after loading parameters. Thanks!

[chidauri]

I hope it is not late enough
You keep everything the same, but use these trained params instead of actually training

[TetianaGryshchuk]

Thank you very much!

[chansonzhang]

thanks, it works for me