Dataloader Iterator breaks and I have no idea why?

Hi,
I’ve set up a Dataset where in the get_item() function, I randomly select a couple of sounds and add them together. After a few batches, my dataloader iterator breaks and I get an error with the message:


AssertionError Traceback (most recent call last)
in ()
----> 1 x, y = next( training_loader_iter)

~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/gluon/data/dataloader.py in next(self)
439
440 assert self._rcvd_idx < self._sent_idx, “rcvd_idx must be smaller than sent_idx”
–> 441 assert self._rcvd_idx in self._data_buffer, “fatal error with _push_next, rcvd_idx missing”
442 ret = self._data_buffer.pop(self._rcvd_idx)
443 batch = pickle.loads(ret.get()) if self._dataset is None else ret.get()

AssertionError: fatal error with _push_next, rcvd_idx missing

I tried to find the cause of the error, but I couldn’t locate anything online. Also, I have no idea what’s happening here and how to debug this error. Could anyone here help?

I tried to recreate a working example of the bug. The thing is, this doesn’t break and works like a charm. So, basically I have no idea where to begin or where the problem is.

random.random( 25)
bsz = 6
toy_data_loader = mx.gluon.data.DataLoader( data, batch_size=bsz, num_workers = 10)
toy_data_loader_iter = toy_data_loader.iter()

toy_idx = 0
max_idx = 1000

while toy_idx <= max_idx:
X, Y = next( toy_data_loader_iter)
print( toy_idx)
toy_idx = toy_idx + bsz

Thanks,
Shrikant

I think the while loop statement should be while toy_idx + bsz <= max_idx) instead of while toy_idx <= max_idx)