Add functional autograd API:hessian

[levi131]

PR types

New features

PR changes

APIs

Describe

1、Add functional autograd API: hessian.
2、Move some helper interfaces together into utils.py.

The API signature is:
def hessian(func, inputs, create_graph=False, allow_unused=False)

This API computes the hessian matrix of func with respect to inputs.
This API is ONLY available in imperative mode.

Parameters:

    func (function): a Python function that takes a Tensor or a Tensor
        list/tuple as inputs and returns a Tensor with a single element.
    inputs (Tensor|list(Tensor)|tuple(Tensor)): the input Tensor or 
        Tensor list/tuple of the function ``func``.
    create_graph (bool, optional): whether to create the gradient graphs
        of the computing process. When it is True, higher order derivatives
        are supported to compute; when it is False, the gradient graphs of
        the computing process would be discarded. Defaults to ``False``.
    allow_unused (bool, optional): whether to raise error or return None if
        some Tensors of `inputs` are unreachable in the graph. Error would
        be raised if allow_unused=False, and None would be returned as
        their gradients if allow_unused=True. Default False.

Returns:

    Hessian (Tensor or a tuple of tuple of Tensors): if function ``func``
    takes a Tensor as ``inputs``, Hessian will be a single Tensor containing
    the Hessian matrix for the linearized ``inputs`` Tensor. If function
    ``func`` takes a Tensor list/tuple as ``inputs``, then the Hessian will
    be a tuple of tuple of Tensors where ``Hessian[i][j]`` will contain the
    Hessian matrix of the ``i``th input and ``j``th input with size ``m * n``.
    Here ``m`` and ``n`` denote the number of elements of the ``i`` th input
    and the ``j`` th input respectively.

TODO:
1、enable test case test_create_graph_true when 3nd order grad op matmul_grad_grad_grad is ok.

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[XieYunshen]

LGTM for set_tests_properties(test_hessian PROPERTIES TIMEOUT 20)

[JiabinYang]

LGTM