I had to read images from the entire zip file because the number of images under a folder exceeds the limit of the Linux system. I used such code
import zipfile import numpy as np from PIL import Image import io import mxnet as mx z = zipfile.ZipFile("train.zip", "r") data = z.read(z.namelist()) data = io.BytesIO(data) im = Image.open(data) im = mx.nd.array(im)
It seems that I get the BGR image which is similar with what I get when using
mx.image.imread(img_path, 1) in gluoncv. However, I have two questions:
- In python3, when using multi gpus, will cause this error:
zipfile.BadZipFile: Bad CRC-32 for file
It seems this problem has somethingt to do with multi processing. I must create a new zipfile.ZipFile for a new image to avoid this problem. Any better solution?
- I am curious about whether such a reading method will be very slow or not. Is there a better way to deal with this problem in mxnet?