Hessian Output

The fminunc and fmincon solvers return an approximate Hessian as an optional output.

[x,fval,exitflag,output,grad,hessian] = fminunc(fun,x0)
% or
[x,fval,exitflag,output,lambda,grad,hessian] = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon)

This topic describes the meaning of the returned Hessian, and the accuracy you can expect.

You can also specify the type of Hessian that the solvers use as input Hessian arguments. For fminunc, see Including Gradients and Hessians. For fmincon, see Hessian as an Input.

fminunc Hessian

The Hessian for an unconstrained problem is the matrix of second derivatives of the objective function f:

$$\text{Hessian }{H}_{ij}=\frac{{\partial }^{2}f}{\partial x_i\partial x_j}.$$

• Quasi-Newton Algorithm — fminunc returns an estimated Hessian matrix at the solution. fminunc computes the estimate by finite differences, so the estimate is generally accurate.

• Trust-Region Algorithm — fminunc returns a Hessian matrix at the next-to-last iterate.

  • If you supply a Hessian in the objective function and set the HessianFcn option to 'objective', fminunc returns this Hessian.

  • If you supply a HessianMultiplyFcn function, fminunc returns the Hinfo matrix from the HessianMultiplyFcn function. For more information, see HessianMultiplyFcn in the trust-region section of the fminunc options table.

  • Otherwise, fminunc returns an approximation from a sparse finite difference algorithm on the gradients.

  This Hessian is accurate for the next-to-last iterate. However, the next-to-last iterate might not be close to the final point.

  The trust-region algorithm returns the Hessian at the next-to-last iterate for efficiency. fminunc uses the Hessian internally to compute its next step. When fminunc reaches a stopping condition, it does not need to compute the next step and, therefore, does not compute the Hessian.

fmincon Hessian

The Hessian for a constrained problem is the Hessian of the Lagrangian. For an objective function f, nonlinear inequality constraint vector c, and nonlinear equality constraint vector ceq, the Lagrangian is

$$L=f+{\displaystyle \sum _i{\lambda }_i{c}_i}+{\displaystyle \sum _j{\lambda }_{j}ce{q}_j}.$$

The λ_i are Lagrange multipliers; see First-Order Optimality Measure and Lagrange Multiplier Structures. The Hessian of the Lagrangian is

$$H={\nabla }^{2}L={\nabla }^{2}f+{\displaystyle \sum _i{\lambda }_i{\nabla }^2{c}_i}+{\displaystyle \sum _j{\lambda }_j{\nabla }^{2}ce{q}_j}.$$

fmincon has several algorithms, with several options for Hessians, as described in fmincon Trust Region Reflective Algorithm, fmincon Active Set Algorithm, and fmincon Interior Point Algorithm.

• active-set, sqp, or sqp-legacy Algorithm — fmincon returns the Hessian approximation it computes at the next-to-last iterate. fmincon computes a quasi-Newton approximation of the Hessian matrix at the solution in the course of its iterations. In general, this approximation does not match the true Hessian in every component, but only in certain subspaces. Therefore, the Hessian returned by fmincon can be inaccurate. For more details about the active-set calculation, see SQP Implementation.

• trust-region-reflective Algorithm — fmincon returns the Hessian it computes at the next-to-last iterate.

  • If you supply a Hessian in the objective function and set the HessianFcn option to 'objective', fmincon returns this Hessian.

  • If you supply a HessianMultiplyFcn function, fmincon returns the Hinfo matrix from the HessianMultiplyFcn function. For more information, see Trust-Region-Reflective Algorithm in fmincon options.

  • Otherwise, fmincon returns an approximation from a sparse finite difference algorithm on the gradients.

  This Hessian is accurate for the next-to-last iterate. However, the next-to-last iterate might not be close to the final point.

  The trust-region-reflective algorithm returns the Hessian at the next-to-last iterate for efficiency. fmincon uses the Hessian internally to compute its next step. When fmincon reaches a stopping condition, it does not need to compute the next step and, therefore, does not compute the Hessian.

• interior-point Algorithm

  • If the HessianApproximation option is 'lbfgs' or 'finite-difference', or if you supply a HessianMultiplyFcn function, fmincon returns [] for the Hessian.

  • If the HessianApproximation option is 'bfgs' (the default), fmincon returns a quasi-Newton approximation to the Hessian at the final point. This Hessian can be inaccurate, similar to the active-set or sqp algorithm Hessian.

  • If the HessianFcn option is a function handle, fmincon returns this function as the Hessian at the final point.