matlab fmincon _ nonlinear optimization

Hi. I have been trying to resolve the following issues, but i cannot solve that.

Used thing : Nonlinear optimization

Used option is

options = optimoptions('fmincon',...

'StepTolerance',1e-10,'Hessian','lbfgs','Display','iter','Algorithm','sqp','Display','iter', 'MaxFunEvals',1e4,'MaxIter',1e4,'FiniteDifferenceType','central',...

'OptimalityTolerance',1e-6)

My question is

1. the objection function value is not changed after certain time.

2. But the first order optimality isnot zero.

3. The lase thing is that (the exitflag = 2) is not good solution ? ( optimality is not zero )

Please check the fig as following.

In my objective function is unknown function. I just known input variable & output. becuase I am using the linearization mode.

So I can only use difference in gradient.

I try to all of thing that i can do, but i cannot solve the problem. Please give me good solution.

( I also using 'interior-point', 'sqp' etc).

Thanks for reading.

[Walter Roberson]

exit flag 2 is not a problem.

Function values not changing for a while is not unexpected as MATLAB hunts around testing more locations in hopes that one of them will reduce the value more. Also, the function value that is displayed does not have the full number of digits, so changes after about the 5th digit or so would not be obvious.

[jisoo jung]

Thank for replying.

So what you are saying is that exit flag doesn`t matter for getting optimal solution. right?

You mean only fval(obj function val) is important ?

And I can not undersatnd 5th digit. I already have detail number.

Could you explain about this ?

[Walter Roberson]

If you look at the example at https://www.mathworks.com/help/matlab/ref/fminsearch.html#bvadxhn-11 you can see lines such as

 Iteration   Func-count     min f(x)         Procedure
     0            1         -6.70447         
     1            3         -6.89837         initial simplex
     2            5         -7.34101         expand

Notice that only about 5 digits after the decimal place are displayed. -7.34101 (for example) would be displayed if the true values changed between (for example) -7.34101493452 to -7.341011415926 to -7.3410105432343 to -7.34100923543564 -- all of those round to -7.34101 for display purposes and yet the scores would be decreasing.

All of the positive exit flags mean that the optimization proceeded as far as was feasible to the limits of the tolerances you set in the options. The exit exitflag value tells you which tolerance became the limit.

[jisoo jung]

So,

Do you understand why the function value is converge but, 1st order optimality is not zero ?

I knew that the 1st order optimality is necessary condition to get optimum & this is gradient of "obj fcn" or " constraint "

[Walter Roberson]

Are you doing a constrained minimization, or an unconstrained minimization? You are using fmincon() which is typically for constrained minimization, with fminunc() being for unconstrained minimization. 1st order optimality is a necessary condition only for unconstrained minimizations.

It is always possible to construct nonlinear equations that have a steep slope near the zero crossing of the gradient, in which case the limit on step size (if only because double precision is a limited number of bits of precision) can force the gradient to be comparatively different from zero for all locations near the true minima.

[jisoo jung]

I have using constrainted optimization.

rellay? .. I get know that 1st order optimality is a necessary condition only for unconstrained minimizations first time. Can I recommened good example about this ?? this is very surprising.

So in my case, most of thing is exitflag = 2 case.

So In this case, how much step toleranece is best ?

[Walter Roberson]

Consider any particular function that has a gradient that at some point is quite different from zero. Now add constraints so that that point is the minima permitted by the constraints. For example, exp(x) has a constrained minima at x = 50 under the constraint that the lower bound on x is 50. The first order optimality could be terrible and yet you might have identified the optimal point under all of the constraints.

So In this case, how much step toleranece is best ?

https://archive.org/stream/personalreminisc00lowr/personalreminisc00lowr_djvu.txt

'One of them asked Lincoln: "How long should a man's legs be in proportion to his body?" and Lincoln replied: "I have not given

the matter much consideration, but on first blush I should judge

they ought to be long enough to reach from his body to the ground."'

In other words, use the tolerances that you need to achieve your goals, but no smaller.

[Bruno Luong]

Walter: "Are you doing a constrained minimization, or an unconstrained minimization? You are using fmincon() which is typically for constrained minimization, with fminunc() being for unconstrained minimization. 1st order optimality is a necessary condition only for unconstrained minimizations."

FMINCON computes the gradient of the Lagrangian and the first ordre condition on the Lagragian is equivalent to KKT condition, which holds regradless if there is constraint or not.

[jisoo jung]

You mean first optimality have to be considered both case?

[Walter Roberson]

Bruno:

Construct any arbitrarily steep function. Now add linear constraints that are satisfied at only one point where the gradient is not close to zero in any direction. The first order optimality becomes irrelevant.

[Bruno Luong]

"You mean first optimality have to be considered both case?"

Yes. Condider this simple 2D example, where I minimize a x(1) on the circle x(1)^2+x(2)^2 = 1 using non-linear constraint:

[x,fval,exitflag,output] = fmincon(@fun,[0; 0],[],[],[],[],[],[],@mycon)

function f=fun(x)
f = x(1);
end

function [c,ceq]=mycon(x)
c = [];
ceq = sum(x.^2)-1;
end

The output of FMINCON is

output = 
  struct with fields:

         iterations: 9
          funcCount: 43
    constrviolation: 5.1070e-14
           stepsize: 2.1720e-07
          algorithm: 'interior-point'
      firstorderopt: 2.2352e-08
       cgiterations: 4
            message: '↵Local minimum found that satisfies the constraints.↵↵Optimization completed because the objective function is non-decreasing in ↵feasible directions, to within the value of the optimality tolerance,↵and constraints are satisfied to within the value of the constraint tolerance.↵↵<stopping criteria details>↵↵Optimization completed: The relative first-order optimality measure, 2.235174e-08,↵is less than options.OptimalityTolerance = 1.000000e-06, and the relative maximum constraint↵violation, 5.107026e-14, is less than options.ConstraintTolerance = 1.000000e-06.↵↵'

The first order reach 2.2352e-08 and trigger the stop of FMINCON (read the message) because the firstorder is rightly computed from the gradient of the Lagrangian.

[jisoo jung]

So , waht do you think aobut my phnomenom ?

The fval is converge. and also constraint also converge to some value.

;But the optimality is very large.

What is your opnion ? and I want to resolve this prob. Please.

My function is unknown function . So I cannot use gradinet personally.

Please give your help.