Segmentation results differ between versions


#1

Could I ask if there is a reason for the output to differ between versions of gluoncv? I’m currently investigating frameworks but am finding that between different versions of gluoncv I am getting different output on the same image sets. For example, on v0.1.0 the same image passed into the exact same demo_fcn.py generates a different output segmentation to mask when ran on v0.2.*

I was hoping to have consistent masks generated so I could create a test suite that will alert me to varying output (I use the mask to process the image); however, it seems that for the same downloaded model I get drastically inconsistent results between GluonCV versions.

Do the different versions download different models, or is it more to do with how GluonCV is operating as it is developed?


#2

It is not the model that was different, but how to predict using it. To get the same dimension it is needed to do prediction multiple times. This makes whole execution slower. So the fix is to do the prediction multiple times as it was before using MultiEvalModel. Take a look into it, if want more details.

I also think that your comment is valid, the behaviour of the demo is strange. So I submitted a PR request to change the demo: https://github.com/dmlc/gluon-cv/pull/175

Nevertheless, gluon-cv project evolves very fast, and there might be occasional breaks in the code, or outdated demos that may require fixing. Community tries its bestto keep it in order, but sometimes things falls into cracks. Also, it is completely open sourced project, so feel free to submit pull requests.