Deep Neural Net for Regression

I am training a NN with 19 inputs and two outputs. I have 1000 random observations, and I am using the first 800 for training, then 100 additional for validation and the remaining for testing. I am using three hidden layers with 15 nodes each and using tanh for actiation function. I have tried to duplicate all the parameters similar to a Python version. The Python version quickly converges to a MSE of about 0.03. However, the Matlab version is giving a MSE around 10 times the error in Python. I have used every parameter same (min batch size = 32, solver = adam, Learning Rate constant at 0.001 etc. Any help is appreciated. The question is why I am not able to train to the same degress of accuracy. I can provide the scripts and data if any one wants.

Answer 1

[Manas]

Hello Pappu Murthy,

I understand that you are facing trouble explaining why the same neural network is converging faster when run in python and not in MATLAB.

There could be multiple reasons as to why this is happening. Since I don’t have the script, data and configurations used in MATLAB and python, I can only speculate as to why this could be happening.

One possible explanation for this behaviour could be the random network initialization before training. Another reason could be that the 5-layer network is not able to provide consistent outputs for this problem. The optimal size and training hyperparameters for a neural network depend on factors such as complexity of the problem and the amount of quality data available. An ablation study would help arrive at a concrete conclusion.

To identify the specific cause of discrepancy, the code and the configurations used in both MATLAB and python implementations need to be compared. Kindly verify if the issue persists after importing the network to MATLAB. Here is a MATLAB documentation on importing networks from external platforms: https://in.mathworks.com/help/deeplearning/networks-from-external-platforms.html?s_tid=CRUX_lftnav

Hope this helps!