After that, we only need to traverse the remaining anchor boxes of A2,A5,A7,A9 and determine whether to assign ground-truth bounding boxes to the remaining anchor boxes according to the threshold.
Why A2, A5,A7,A9?
I think it should be the anchors except the A2,A5,A7,A9. Also the matrix A should only eliminate the row in the mean while keep the column values. If you try to reimplement the code, you will found out.