Does mxnet allow to compute higher order differentiation like in tensorflow, pytorch? Because in their official documentation it’s written that only few operations are available for higher order differentiation.

# Higher order differentiation

**thomelane**#2

Support for higher order differentiation is limited at the moment, but coverage is improving.

You should check out the `mxnet.autograd.grad`

function for this.

```
x = nd.array([1, 2, 3])
x.attach_grad()
with autograd.record():
y = nd.exp(x)
y_grad = autograd.grad(y, x, create_graph=True, retain_graph=True)[0]
y_grad.backward()
print(x.grad)
```

Simplest way to see if operators are supported is to try it, but you can tell by looking at the backend source code if you’re not scared of C++ code. One sign that an operator supports higher order gradients is that the `FGradient`

attribute is not registered to a backward operator, e.g. exp. And an indicator that an operator doesn’t support higher order gradients is that the `FGradient`

attribute is registered to a backward operator, and the backward operator itself doesn’t register another `FGradient`

function, e.g. sigmoid.

2 Likes