The idea of the Quasi-Newton method is to update the approximation of the Jacobian matrix using the information from the function values and the change in the unknowns from the previous iteration. The method updates the Jacobian approximation using the Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula or the DFP formula, which are two common formulas used in the Quasi-Newton method.
The Quasi-Newton method starts with an initial guess for the solution. Then, it iteratively improves the guess until it converges to the solution. At each iteration, the method computes the function values and updates the approximation of the Jacobian matrix.
Here are the steps to apply the Quasi-Newton method:
- Choose an initial guess for the unknowns, x0.
- Set the approximation of the Jacobian matrix, B0, to the identity matrix or another suitable positive definite matrix.
- For each iteration k=0,1,2,..., do the following:
a. Compute the function values, f(xk).
b. Terminate the algorithm if the norm of the function values is smaller than a specified tolerance.
c. Compute the change in the unknowns, s = xk+1 - xk.
d. Compute the change in the function values, y = f(xk+1) - f(xk).
e. Update the approximation of the Jacobian matrix using the BFGS or DFP formula.
f. Update the guess for the unknowns, xk+1 = xk + pk.
g. Terminate the algorithm if the change in the guess for the unknowns is smaller than a specified tolerance.
h. Go to step 3.
4. Output the final solution, x*.
Note that the Quasi-Newton method may not converge or may converge to a local minimum if the function is highly nonlinear or has multiple local minima. Therefore, it is important to check the convergence and the correctness of the solution, you can also refer to the following for more information.
