I have been exploring the Neural Network capabilities of Matlab concerning fitting/interpolation, using the nftool.
So far I have been using a vector of:
- [N x 2] as an input
- [N x 1] as an output
where N is ~40000. Training is rather fast (say within 10 minutes) and with hiddenLayerSize of [4,10,20] I get very good agreement when testing the network; effectively the R parameter is unity (see test.png).
Training parameters are:
net.divideParam.valRatio = 5/100;
net.divideParam.testRatio = 5/100;
net.divideParam.trainRatio = 1 - net.divideParam.valRatio - net.divideParam.testRatio;
using Bayesian regularisation.
Now, I need to repeat the same process with an input vector of [N x 3] and output again [N x 1], but now N is in the order of 4*10^6 (there is the possibility to expand even beyond that, with N x 6 input and even larger Ns). It is still possible to get a good training by increasing No of layers (I have obtained good performance with hiddenLayerSize = [20, 5, 10, 20];), though it is much more time consuming (takes around a day to train) and whereas the match is good (again R~1), there are clearly some points that do not get captured as shown in the regression plot and there is a bit of noise when the trained network is used; this I dont think is from over-fitting, as performance tends to improve with increasing layer sizes.
Is there any way to improve the performance of training? Can deep learning networks or convolutional networks help here? It is a bit unclear if it is possible to use them for fitting applications, as they are shown in Matlab examples..
Any idea is welcome