You can use any MATLAB code to implement and accelerate your objective function and constraint functions, including any code like gpuArray that invokes the GPU.
Other than the part supplied by you, however, I'm not aware of anything in the Optimization Toolboxes that would use the GPU. There is the 'UseParallel' option parameter that you can use to make behind-the-scenes operations in the optimization algorithms take advantage of the Parallel Computing Toolbox, if you have that toolbox as well. However, as far as I'm aware, that will only take advantage of multi-core CPUs and not the GPU.
I wonder to myself whether there is even any behind-the-scenes functionality that is sufficiently parallel to benefit from the GPU. Some of the the linear equation solving done to calculate search directions might, I suppose. On the other hand, that calculation would need to transfer data to/from the GPU and could incur significant communication overhead, except maybe on very recent graphics cards.
