I work for practical deployment of cnn model, and I want to check a specific desired class accuracy of my deep-learning model (image-classification) on my validation set. Model is trained on a subset of IMAGENET-2012 dataset. A simple template code like score.py for this might be useful for me.
Thanks
When you’re interested in an individual class, precision and recall may be better metrics to look at. You can, however, define accuracy as the number of correctly classified items (correct being defined as “if label is class A, prediction is class A and if label is not class A, prediction is not class A”) divided by the total number of items. I modified mxnet.metric.Accuracy
to implement a SingleClassAccuracy()
as an example:
class SingleClassAccuracy(mx.metric.EvalMetric):
def __init__(self, axis=1, name='single_class_accuracy',
output_names=None, label_names=None):
super(SingleClassAccuracy, self).__init__(
name, axis=axis,
output_names=output_names, label_names=label_names)
self.axis = axis
def update(self, label, preds):
"""Updates the internal evaluation result.
Parameters
----------
label : int
The class index
preds : list of `NDArray`
Prediction values for samples. Each prediction is a vector of likelihoods for all classes.
"""
for pred_label in preds:
pred_label = nd.argmax(pred_label, axis=self.axis)
pred_label = pred_label.asnumpy().astype('int32')
self.sum_metric += (pred_label.flat == label).sum()
self.num_inst += len(pred_label.flat)
acc = SingleClassAccuracy()
acc.update(2, [nd.array([[0, 0, 1],[0, 0, 1], [0, 1, 0], [1, 0, 0]])])
print(acc.get())
1 Like