Hey guys check out my new benchmark for mxnet vs tensorflow. And I’ve made a decision that I am gonna use mxnet as long as possible. Wanna know why? Have a look yourself.
You should use deeper models with longer training periods, 30 secs means a big percentage is on initialization, which will be a much smaller fraction in an actual setting (training times of several hours). Also, wall time is a bad comparison metric. And multi-gpu is also more relevant for practitioners, where I heard is where tensorflow starts to shine.
Just my 2 cents.
We can definitely do extreme deep badass benchmark but my goal was to see which one is faster in general, because most of the time when we code we really don’t use all strategies that comes with a framework to increase performance, we just build our model and train it in one go and that’s the case where I wanted to compare these frameworks. What you are talking about is more of like production and deployment in which I am not interested. Even though if we do that kinda benchmark… Mxnet would be the clear winner definitely, it’s more distributed and scalable, more so than tensorflow. FYI check out the recent Nvidia’s benchmark which is more of like what you are saying:
1 Like
I’ve just used time.time() rather wall time as you said and still got the same results.
One very specific thing that I don’t like about tensorflow is it’s bad convergence. In general pytorch has the best loss convergence I’ve seen, and after that mxnet.