I was using a Symbol block and had mx.symbol.Embedding layer which worked without any issues. When I changed it to sparse.Embedding I realized that SymbolBlock doesn’t seem to work with row_sparse.
weight = mx.symbol.var("encoder_weight", stype='row_sparse') embed = mx.symbol.sparse.Embedding(data=data, weight=weight, input_dim=input_dim, output_dim=embedding_size, name='embed', sparse_grad=True)
From the code in gluon/block.py it looks like SymbolBlock is inherited from HybridBlock and which intern is from Block. None of them check parameter storage type.
Is there specific reason for SymbolBlock not supporting row_sparse storage type ?
# check if any symbol is row_sparse row_sparse_storage = ndarray.ndarray._STORAGE_TYPE_STR_TO_ID['row_sparse'] for i in out: for j in i.get_internals(): assert(j.attr("__storage_type__") != str(row_sparse_storage)), \ "SymbolBlock doesn't support Parameter '%s' because its storage " \ "type is 'row_sparse'." % j.name