RNN implementation difference between Gluon and Symbolic

Gluon
1

I have a few questions regarding RNN (LSTM/GRU) implementation in Gluon and/or Symbolic.

  • Is there a difference between how LSTM and GRU are implemented in Gluon vs Symbolic?

  • Does Gluon follow the CudNN RNN implementation for LSTM/GRU?

  • In case that’s true, how does it handle packing the input elements into contiguous memory? PyTorch has a method called pack_padded_sequence, does Gluon do something like this internally?

  • Does Gluon RNN implementation support variable length input? In case it does, what’s the way to pass a variable length NDArray to it since MXNet NDArray requires elements to be of same dimensions?

2
  1. no
  2. for GPU, yes. CPU, no
  3. Gluon currently requires all samples in a batch to have the same length. You need to pad them to the same length yourself.
  4. not within the same batch
3

Thanks a lot Eric. :slight_smile: I just have one final question. About point no. 3, is just padding them to same length sufficient or do we need to do something to make them contiguous in memory?

4

We heard that the advantage of the imperative style of mxnet over symbolic style (tensorFlow) is allowing variable length sequence.

But from the doc bucketing, it seems that mxnet support variable length sequence input only within the symbolic style (module)?

It would be great if Gluon support it.