Calculation of the gradient of a function in BFGS Quasi-Newton Algorithm

I have a function that I want to minimize using the BFGS Quasi-Newton algorithm.

If .

are given.

Also, A is given, B is not given, and C is a function of B only.

If I want to calculate to the gradient of the function (). Should I calculate it with respect to all variables or with respect to B only?

Answer 1

[Matt J]

With respect to all the variables that are unknown.

[Ahmed Nasr]

which is B only or B and C? (C is a function of B)

[Matt J]

You should take the gradient with respect to both B and C and then apply the Chain Rule:

[Ahmed Nasr]

I have a constraint that B*C > 0 .. But, I usually in my algorithm code substitute C by its function of B and take the gradient over B only as after replacing C with its function of B, B is considered as the only unknown variable in the function. Is that true?

[Matt J]

It can be true. Or you can treat B and C as separate unknowns subject to an equality constraint,

C-g(B)=0

where g() is the function relating C to B. But it may make the optimization algorithm slower if it has to deal with additional variables and constraints.