Multi task learning (The accuracy tested of training dataset is not as high as training accurracy )

Discussion
1

Trained a multi-task model， the training accuracy as below:

image
image.png888×652 17.1 KB

Two different tasks’ accuracy are high, however , Task1’s accuracy can only reach 90% when tested in training data set.
And the prediction code as below,

import mxnet as mx
import numpy as np
import cv2
import time
import glob
from collections import namedtuple
Batch = namedtuple(‘Batch’, [‘data’])

ctx = mx.gpu(0)
sym, arg_params, aux_params = mx.model.load_checkpoint(’/root/insight_multiTask_thres/src/thres_none’, 52)

all_layers = sym.get_internals()

sym = all_layers[‘fc3_output’]

mod = mx.mod.Module(symbol=sym, context=ctx, label_names=None)
mod.bind(for_training=False, data_shapes=[(‘data’, (1,3,112,112))])

mod.set_params(arg_params, aux_params,allow_missing=True)

file_list = glob.glob("/root/emotion_data/surprise/*")
ccount = 0.0
rcount = 0.0
for file in file_list:
img1 = cv2.imread(file)
img = cv2.resize(img1,(112,112))
nimg = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
aligned = np.transpose(nimg, (2,0,1))
input_blob = np.expand_dims(aligned, axis=0)
data = mx.nd.array(input_blob)

start = time.clock()
db = mx.io.DataBatch(data=(data,))
mod.forward(db, is_train=False)
prob = mod.get_outputs()[0].asnumpy()
#print(mod.get_outputs())
pred = np.argsort(prob[0])[::-1]
# label_id (same as label if MNIST), score

prob2 = mod.get_outputs()[1].asnumpy()
pred2 = np.argsort(prob2[0])[::-1]

# prob3 = mod.get_outputs()[2].asnumpy()
# pred3 = np.argsort(prob3[0])[::-1]

end = time.clock()
print(str(pred[0])+"  "+str(pred2[0]))

ccount = ccount+1
if pred[0] == 7:
    rcount = rcount+1

print(rcount/ccount)

Thanks anyone who can help me.

2

Hi @sljiangsl,

I’m not sure I understand your issue. From the logs provided, it looks like the training accuracy of task 1 is consistently >99% (multi-accuracy-task1). Are you talking about the accuracy on the test set? If so, are you trying to replicate some other results? What is the target accuracy you are expecting?

3

Thanks for your reply, I just test on the training data set to check my my model and multi-task prediction code, but when I run the prediction code in the training data set , the accuracy just reach 0.90. It is too weird because I think the test accuracy in training data set is the same as the training accuracy, maybe my prediction code of multi-task is not right?

4

Sounds like you’re doing different pre-processing or post-processing steps for the data being passed to the model for training and inference. I once forgot to apply image normalization to the images for inference, and the accuracy was much worse for the same data in training and inference.

As a sanity check, log an example of the input being passed to the model for training, and then the same for inference. Check the ranges of values for example, to see if they look like samples from the same distribution. And check your post processing steps (e.g. non maximal suppression for object detection) by applying the training and inference post-processing steps to the same model output.

5

hello.
I am try to building a network with multi task learning, but I do not how to write the building and training code with mxnet symbol.
Can you send me your code about them?
thank you.