The following code is for 1- cosine similarity loss, is it correct? Any advice will be appreciated, thanks.
1-cosine similarity loss
head_feat = mx.symbol.reshape(head_feat, shape=(-1, ), name='head_feat_reshape')
head_feat_vis = mx.symbol.reshape(head_feat_vis, shape=(-1,), name='head_feat_vis_reshape')
dot = mx.symbol.dot(head_feat, head_feat_vis, name='dot')
head_feat_l2 = mx.symbol.norm(head_feat, name='head_feat_l2')
head_feat_vis_l2 = mx.symbol.norm(head_feat_vis, name='head_feat_vis_l2')
head_feat_dot = mx.symbol.dot(head_feat_l2, head_feat_vis_l2, name='head_feat_dot')
cos_sim = mx.symbol.elemwise_div(dot, head_feat_dot, name='cosine similarity')
constant1 = mx.symbol.ones(shape=(1, ), name='constant_1')
cos_sim = mx.symbol.elemwise_sub(constant1, cos_sim, '1_cosine _imilarity')
cos_loss = X.loss(cos_sim,
grad_scale=1.0 / batch_roi * scale_loss_shift,
name='cos_loss')