For example using the latest mxnet installed using pip install mxnet --pre
You could load the symbols into Gluon.
net = get_symbol()
net.save('inception-v4-symbol.json')
gluon_net = mx.gluon.nn.SymbolBlock.imports('inception-v4-symbol.json', ['data'])
gluon_net.initialize()
gluon_net(mx.nd.ones((1,3,299,299)))
And then use a normal training loop to load your data. Make sure they are big enough though, 299x299 seems to do the trick.
However it looks like you would need to be quite careful with how you initialize the parameters:
Have a look at this question to get an idea of how to do that granularly.