I was using a Symbol block and had mx.symbol.Embedding layer which worked without any issues. When I changed it to sparse.Embedding I realized that SymbolBlock doesn’t seem to work with row_sparse.
weight = mx.symbol.var("encoder_weight", stype='row_sparse')
embed = mx.symbol.sparse.Embedding(data=data, weight=weight, input_dim=input_dim,
output_dim=embedding_size, name='embed', sparse_grad=True)
From the code in gluon/block.py it looks like SymbolBlock is inherited from HybridBlock and which intern is from Block. None of them check parameter storage type.
Is there specific reason for SymbolBlock not supporting row_sparse storage type ?
# check if any symbol is row_sparse
row_sparse_storage = ndarray.ndarray._STORAGE_TYPE_STR_TO_ID['row_sparse']
for i in out:
for j in i.get_internals():
assert(j.attr("__storage_type__") != str(row_sparse_storage)), \
"SymbolBlock doesn't support Parameter '%s' because its storage " \
"type is 'row_sparse'." % j.name