Basics of fminsearch to be explained in laymens terms

Could someone explain me in layman's terms this function? For instance, in the example:

X = fminsearch(@sin,3)

Thank you, Gaha

Answer 1

[Amit]

In layman's term, fminsearch tries to find the minima of the function starting with the point given. Like in your case, fminsearch(@sin,3) or fminsearch(@(x) sin(x),3) tries to find the minima for sin function. The output will be the point where the minima is found. As you know, sin has minima at -1, it will find one of those values for x.

It can optimize for multivariable scenario as well. However, in fminsearch, you can not input the bounds for the variables.

Hope this is clear!

[Nuchto]

Thanks, getting there! Two questions still: 1. What is the role of the "3" in the formula above? and 2. what values is Matlab trying to find the minimum? Is it random? For instance, in the function [given matrix]*X-[given matrix2], you tell Matlab an initial valur for X, but from there, what values does it try? Hope I made myself clear...

[Amit]

In the image here, you see two minima. fminsearch attempts to finds the value of the variable, where one of such minima is. You provide a initial value (in this case, its 3) from which the search starts. Depending on the initial value, you can get any of these minima. (Fminsearch does not guarantee a global minima).

For your case, you know the value of sin function is -1 at many values of x. For sin function, the minima will exist where sin(x) is -1. Try doing this: use fminsearch for sin function and give different values instead of 3 and see what you get. You'll get a different number, however all those number will a minima in that neighborhood.

[Nuchto]

I see, thanks for this explanation. Then, is it possible to not give a value from which to start? I don't quite understand why the need for giving this value.

[Amit]

You do need a value to start. If you dont care, you can give a random number to start with. You just have to be careful because if the function have more than one minima, then the final answer will depend on the starting point.

Also, there are other functions that you can look through - like fminbnd This actually only minimizes for a single variable function. You dont give the inital point, however you'll have to give the lower and upper bound (the range in which you expect the minima to fall).

[John D'Errico]

Suppose I asked you to locate the lowest point on earth? Worse, imagine that you are blind, and can only tell what direction is locally up or down by testing with your cane? You can walk around from a starting point, moving locally down hill. Eventually when you find yourself at the very bottom of a trench and can find no direction that moves downhill, you stop, declaring the operation a "success".

This is all fminsearch does, starting with the point that you give it. If our blind searcher starts out in an area where they can walk down hill to the globally minimum point, then they will succeed in their given quest.

How could fminsearch know how to find a minimum of a completely general black box function without you giving it a starting value? Computers are not the magical things that we see in the movies and TV shows, at least not yet.

If you have these questions, it is surely time to start reading about the topic.

[Amit]

Nice example John.

[Nuchto]

Nice example indeed, John. Thanks, Amit, again. Just last thing. My question was also about what kind of algorithm does Matlab use to seach (from that starting condition [value]). So if it it a transformation matrix that I am looking for, and give as the initial value an IM, what will be Matlab's next try? Just randomly tries different matrices and gives as an output the one that minimizes the function? Or does it have an strategy?

[Jos (10584)]

The search is done via a method called "Nelder-Mead simplex direct search". Here is some more information: http://www.scholarpedia.org/article/Nelder-Mead_algorithm

This method does not take derivatives (slopes) but just evaluates function values. In that sense, the word "downhill" in John's great example is a little misleading. The search is not taking the slope of the function into account, it just looks whether a level is lower or higher than another level.

[Nuchto]

That's great, thank you very much to all of you!.