I am new to mxnet. I am working with a dataset for which I know that the model and the data allow a good classification.
I observe the following phenomenon: I am training a simple feed forward network using mx.model.FeedForward.create reaching an accuracy in the 80s. If I apply the returned model to the traning data and check the confusion matrix as well as the accuracy, it’s just about 3%.
I tried the mxnet examples (on mxnet.apache.org) to assure that my installation in R works correclty.
I’ve searched the web a lot and tried all different options but can’t find what the issue is. Hopefully somebody can help.
Thanks in advance for your replies!
I am pasting the code that I am using here. Is there any way to attach some files? I’ve got an .r-file and an .RData so that people could try to redo the example on their own:
require(mxnet) # optional. Just for visualization of confusion matrix require(caret) nn.layer.input <- mx.symbol.Variable('data') nn.layer.fc_3 <- mx.symbol.FullyConnected(data = nn.layer.input, num_hidden = 4) NN_model <- mx.symbol.SoftmaxOutput(data = nn.layer.fc_3) mx.set.seed(0) devices <- mx.cpu() model <- mx.model.FeedForward.create(NN_model, X = data.x.train, y = data.y.train, ctx = devices, array.batch.size = 32, optimizer = "adam", begin.round = 10, num.round = 50, learning.rate = 1e-3, eval.metric = mx.metric.accuracy) prob.y.predict <- predict(model, data.x.train, array.batch.size = 1) y.predict <- max.col(t(prob.y.predict)) table(y.predict, data.y.train) #Here you can already see that there is too much of confusion for being in the 80s # optional if caret is available confusionMatrix(factor(y.predict, 1:4), factor(data.y.train,1:4))