Dropout on sparse input

1

The mx.symbol.sparse package does not support adding a Dropout layer on a sparse input. Well, technically, it does, but the input is being converted to a dense representation slowing down training considerably.

Is there a reason this is not supported? Any alternatives?

2

Yes, dropouts are currently not supported yet by the mx.symbol.sparse package. In order to avoid the conversion, you could create your own operator https://github.com/apache/incubator-mxnet/blob/master/example/numpy-ops/custom_sparse_sqr.py#L22-L36

3

Thanks. I’ll give that a shot.