fmincon with interior point method vs CasADi with IPOPT

I have a question about the solver differences between MATLAB's fmincon interior-point solver and CasADi's IPOPT solver. From what I understand, they both fall under the class of interior-point methods, where IPOPT is said to be better suited for high-dimensional optimization problems. I have implemented an MPC problem for autonomous vehicle motion planning with both fmincon and CasADi, using the interior-point method and IPOPT, respectively.

Problem Setup:

• The MPC problem is formulated in the Frenet coordinate frame, where the states are:
• Longitudinal distance from the road's center.
• Lateral distance from the road's center.
• Heading angle.
• Speed.
• The control inputs are acceleration and steering angle.
• The neural network model used is a feedforward neural network with:One hidden layer consisting of 512 neurons and tanh activation.
• Inputs: 4 state variables and 2 control inputs.
• Outputs: Derivatives of the states, which are integrated using Euler integration to obtain the next state.
• Collision avoidance inequality constraint with Euclidean distance after transforming the Frenet state to a Cartesian state.

Warm Start: The first MPC problem at timestep 1 is initialized with the available nominal trajectory to be tracked. Subsequent MPC problems are warm-started with solutions shifted one step ahead from the previous timestep's MPC solution.

Solver Details:

• fmincon: Provided with analytical gradients and Jacobians of both the cost function and the neural network equality constraint.
• CasADi (IPOPT): Uses automatic differentiation based on a computational graph (please correct me if this understanding is incorrect).

Observations:

• CasADi with IPOPT is faster in computation time but struggles to converge to a cost value comparable to fmincon.
• The problem is non-convex due to the neural network equality constraints and Euclidean distance inequality constraints for collision avoidance.

| Horizon | Method | Cost Value | Average Computation Time for All MPCs (s) |

| 10 | CasADi | 26,132.95 | 0.143 |

| 10 | fmincon | 17,912.42 | 8.162 |

| 20 | CasADi | 47,528.11 | 2.356 |

| 20 | fmincon | 18,852.66 | 23.57 |

Questions for Discussion:

1. Why might IPOPT (CasADi) struggle to converge to a cost value similar to fmincon despite being faster?
2. Does automatic differentiation in CasADi have limitations when dealing with non-convex constraints, especially those involving neural networks?
3. Are there recommended settings or strategies to improve the convergence performance of IPOPT in non-convex MPC problems?

Notes:

• I am willing to share implementation code for both solvers if anyone needs a more detailed analysis.
• Any insights or recommendations would be greatly appreciated!

Thanks in advance for your time and help!

[Torsten]

If you are certain that the implementation of your problem is correct for both solver environments, why don't you just accept that two different solvers behave differently ? What's the purpose of searching for an explanation ? Do you want to alter parts of the solvers or parts of your problem formulation ?

[Iman Askari]

Torsten,

Thanks for the comment. I have made sure that the setting and information provided to both fmincon and CasADi are consistent to the best of my ability. I am searching for further explanation because I am using both methods to benchmark my method of solving the problem for a journal response submission. I am trying to understand the differences as much as possible because in the earlier version of my manuscript, I had just compared my method with fmincon, and the reviewer's comments were that I need to compare with other state-of-the-art solvers. I then chose CasADi with IPOPT as another benchmark solver. Now, for the revised manuscript and review response, I would like to have a clear picture of both solvers in case a reviewer has further comments about this performance difference. Since MATLAB's documentation does not provide enough details on the implementation details of fmincon's interior-point method, I could not figure out why this happens. I would appreciate any insights.

Thanks!

[Torsten]

Maybe you should start with a less complicated problem for a comparison. With the actual level of complexity, it will be almost impossible to find a deeper reason for the different behaviours - especially because the solvers don't converge to the same solution.

It would be interesting to know the results of your method of solving in this context.

[Iman Askari]

Thanks for the recommendation. I however would not want to simplify the problem as the premise of my approach is aruged based on such difficlt optimization problems involving neural network state-space models.

My method is based on reformulating the MPC problem as a Bayesian inference problem and solving the inference using an efficient particle filter/smoother method I have developed. The preprint is availible at: https://arxiv.org/abs/2310.08045 if you are interested. But, my argument for the approach is that the propsoed method would perform better in such noncovex problem of using neural network state-space models. My computation time is better by orders of magnitude when compared fmincon and CasADi.

[Torsten]

I however would not want to simplify the problem as the premise of my approach is aruged based on such difficlt optimization problems involving neural network state-space models.

What I wrote is that I would start with a simpler problem. I think it's hopeless to compare codes with such complex problems if the aim is to analyse and explain different behaviour. If you simply wanted to describe and contrast reliability and performance, the difficulty of the problem wouldn't matter.

But there are other forum members around who have more experience with the topic of MPC. They might be able to comment more specifically than I can.