Hi, All,
Would like to get a general purpose attention layer implementation in MxNet if possible.
Any link/guide is appreciated.
EDIT:
I know sockeye has quite some models but it requires a whole package while I would prefer a very simple layer if possible.
Thanks,
There is a GluonNLP toolkit has been introduced recently. It is a separate package that works using MXNet. It has an AttentionCell as a part of it.
Here is the Github - https://github.com/dmlc/gluon-nlp
Please notice, that it works only with master
version of mxnet, so you will have to update to pip install --pre --upgrade mxnet
before you can start using NLP Toolkit.
Appreciate this. I was informed once of this existence but didn’t pay attention. It’s really nice to have this in place though it could use more documentation and example