I apologize in advance for any inconvenience as this is my first post.
I am trying to train an SSD Model from GluonCV on a custom dataset, created as an LSTRecord
I am referencing:
I am encountering an issue whereby the SmoothL1 metric used in  is reporting Nan; my model
is unable to detect my target object in a preliminary test.
To diagnose the issue, I tried printing out the anchor boxes generated by this snippet of code in :
def get_dataloader(net, train_dataset, data_shape, batch_size, num_workers):
from gluoncv.data.batchify import Tuple, Stack, Pad
from gluoncv.data.transforms.presets.ssd import SSDDefaultTrainTransform
width, height = data_shape, data_shape
# use fake data to generate fixed anchors for target generation
_, _, anchors = net(mx.nd.zeros((1, 3, height, width)))
batchify_fn = Tuple(Stack(), Stack(), Stack()) # stack image, cls_targets, box_targets
train_loader = gluon.data.DataLoader(
train_dataset.transform(SSDDefaultTrainTransform(width, height, anchors)),
batch_size, True, batchify_fn=batchify_fn, last_batch=‘rollover’, num_workers=num_workers)
train_data = get_dataloader(net, dataset, 512, 16, 0)
the anchors were reporting Nan in the last dimension of the bounding box coordinate e.g.
[48.437504 29.437502 10.88711 nan]
Is anyone able to advise on a way to resolve this NaN issue?
As an interim solution, I am looking to generate the anchor boxes in the manner described in  but it lacks the OHEMSampler used by the SSDTargetgenerator, located in SSDDefaultTrainTransform which i am concerned might affect my model’s performance.