MXNet Forum

Feeding sparse data to DataLoader


#1

Is there a way to feed sparse data to a gluon DataLoader?
In a public gluon SageMaker example, this is done via creating an ad-hoc class:

class SparseMatrixDataset(gluon.data.Dataset):
    def __init__(self, data, label):
        assert data.shape[0] == len(label)
        self.data = data
        self.label = label
        if isinstance(label, ndarray.NDArray) and len(label.shape) == 1:
            self._label = label.asnumpy()
        else:
            self._label = label       
        
    def __getitem__(self, idx):
        return self.data[idx, 0], self.data[idx, 1], self.label[idx]
    
    def __len__(self):
        return self.data.shape[0]        

Is there a simpler way?
Cheers


#2

Take a look into this example. They use mxnet.io.LibSVMIter, but there is a comment that it is an experimental feature and should be used with care.