Attention Mechanisms in GluonNLP


Does anyone have working examples of how to use the Attention cell in the gluon NLP package? How general purpose is it and can they be stacked to make a hierarchical attention model?


Hi @dustin.holloway,

You can find an example usuage of the AttentionCell in the Google Neural Machine Translation System example found here.

Specifically you’ll want to look at this line for usage. And the selection of the different types of cells is defined here.

@Sergey has been working on a model using attention so maybe able to give some additional advice if you need it.