Shuffle different on OSX vs Ubuntu


#1

I think I’ve discovered a discrepancy in the behavior of mxnet.random.shuffle() on OSX and Ubuntu. Both systems are running mxnet 1.3.1.

Minimal reproducible example:

import random as rnd
import numpy as np
import mxnet as mx
import mxnet.ndarray as nd

rnd.seed(1)
np.random.seed(1)
mx.random.seed(1)
mx.random.shuffle(nd.arange(10))

OSX 10.12.6 Output

[9. 3. 6. 7. 1. 8. 4. 2. 0. 5.]
NDArray 10 @cpu(0)

On Ubuntu 16.04.5 LTS

[3. 5. 4. 2. 6. 9. 7. 0. 8. 1.]
NDArray 10 @cpu(0)

This can have serious repercussions for SGD if you use shuffle() to shuffle the data. If results are not reproducible across operating systems, that would be bad.

== Ben


#2

This is not a bug. You cannot rely on the random seed being the same on different machines, given that the underlying random number generators may be different.


#3

Interesting. But then why are the results identical with Numpy’s random number generator?

np.random.seed(1)
np.random.permutation(np.arange(10))

OSX:
array([2, 9, 6, 4, 0, 3, 1, 7, 8, 5])

Ubuntu:
array([2, 9, 6, 4, 0, 3, 1, 7, 8, 5])

One would think that mxnet could do the same.

== Ben


#4

You’re only using mx.random.seed in the above MXNet code (the other seeds don’t do anything)
From the docs of MXNet (https://mxnet.incubator.apache.org/_modules/mxnet/random.html)

    Random number generators in MXNet are device specific.
    `mx.random.seed(seed_state)` sets the state of each generator using `seed_state` and the
    device id. Therefore, random numbers generated from different devices can be different
    even if they are seeded using the same seed.

Note the seed in MXNet is generated with the device id as well as the seed state in contrast to Numpy’s random seed behaviour which an identical seed results in identical random numbers.

Feel free to re-open this issue: https://github.com/apache/incubator-mxnet/issues/7025 as it has come up.

Vishaal


#5

I see. So this is expected, if not ideal, behavior. Thanks!

== Ben


#6

I still think it would be good to provide consistent behavior across platforms. Checking the code I found that it uses a different shuffle method depending on the platform https://github.com/apache/incubator-mxnet/pull/10048/files#diff-e33de60345ba28d085b755ab6de3691eR50


#7

I think the documentation means that:
mx.random.seed(0) may generates different numbers on GPU(0) and GPU(1). That’s fine. But if I change the platform and compare the result on GPU(0), they are expected to be consistent. No?


#8

Actually if i do mx.nd.random.uniform(0,5, shape=(5,5)), the result is consistent across OSX and Linux