Why does fmincon give different results in code gen vs. MATLAB for a numerically stiff problem with non-linear constraints?

I use fmincon to solve a 27 variable problem with 27 nonlinear equality constraints and some lower/upper bounds. When I do codegen for my code I get different results from fmincon than I do in base MATLAB. What is going on?

Answer 1

[MathWorks Support Team]

For how some optimization problems are defined, this can be due to a minor difference between the optimization solver in code generation vs. MATLAB. We recommend solving this by editing how the optimization problem is defined.

Detailed Explanation:

As general background, understand that in order to make progress, the fmincon SQP solver must linearize the nonlinear objective and constraints into a quadratic programming problem as described in our documentation

https://www.mathworks.com/help/releases/R2020a/optim/ug/constrained-nonlinear-optimization-algorithms.html#bqa_td3

For some optimization problems, MATLAB's fmincon SQP algorithm always yields a full rank linearization of the nonlinear equalities. This makes it easy to solve the problem. In the code generation version, however, a slightly different path is taken due to round-off error, causing the equality constraint linearizations to becomes degenerate. Fmincon determines that the linearized problem might still be solved if some linearized equalities are removed. These are removed and the solver continues. After additional computation, the solver realizes that removing those equalities introduced unforeseen numerical issues. The linearized problem is relaxed in an attempt to recover. This relaxation allows the SQP solver to get stuck in a numerically degenerate region.

Ultimately, the main issue here is that the optimization problem as defined is numerically stiff in certain areas of the feasible region. The MATLAB fmincon solver is "lucky" in that it dodges a degenerate region but the codegen version does not.

Solution 1: The preferred solution is to change the way the optimization problem is defined. First, re-evaluate the model to ensure these kinds of degeneracies are expected. We also recommend using exact gradients for the constraints and objectives if possible. Doing so makes it more likely that the MATLAB and codegen behaviors will match up. You can see examples here:

https://www.mathworks.com/help/releases/R2020a/optim/ug/nonlinear-constraints.html#bskkri8

https://www.mathworks.com/help/releases/R2020a/optim/ug/writing-scalar-objective-functions.html#bsj1e55

Solution 2: Alternatively, if exact gradients are too time-consuming to compute, you could try reducing the step size of the finite difference approximation. This involves manipulating the "FiniteDifferenceStepSize" or "TypicalX" values. 

However this option is not recommended for deployment, because

A) This solution is application dependent and may not generalize to other situations (i.e. it might fix this case but still fail in others)

B) this is more resource intensive due to more iterations.  Solution 1 is preferred because it is likely more robust and resource efficient.