Bad_alloc when I use mx.io.ImageRecordIter

Discussion
1

I try to get Image Data with ImageRecordIter, but it caused python aborted.

image
image.png1757×307 17.6 KB

Did it happen because my memory is not enough? I ran this on vultr with 1 GB memory.
Can anybody figure this out? Or how many memory should I get to use ImageRecordIter?

2

Yes, you can get std:bad_alloc when you run out of memory. I am surprised that data of this size gave you an error, when each batch should be approx 5mb (assuming Float32), even with overhead. Were other processes using the memory at the time?

See if the following also gives you the same error
after downloading some test data; wget http://data.mxnet.io/data/cifar10/cifar10_val.rec in bash.

import mxnet as mx
data = mx.io.ImageRecordIter(path_imgrec="cifar10_val.rec", data_shape=(3,32,32), batch_size=394, prefetch_buffer=1)

If it doesn’t give you the same error, there might be an issue with your record file. You can also try the same with Gluon’s ImageRecordDataset.