Wrong prediction results from feedforwardnet

Question

0 votes

I made a simple feedforward net as follows:

mynet = feedforwardnet(5)
mynet.layers{1}.transferFcn = 'poslin'; % one hidden layer(5 neurons) with poslin = ReLU activation function
mynet.layers{2}.transferFcn = 'purelin'; % last layer has simply linear activation function 

I want to train this Neural Network to learn a non-linear function that looks like this. So basically it is a regression problem.

So we have two inputs(u1, u2), and one output(y).

After training I get the weights and biases by:

W1 = mynet.IW{1,1}; b1 = mynet.b{1}; W2 = mynet.LW{2,1}; b2 = mynet.b{2}

Now to estimate the output, we can simply use:

inputs = [3;2] % u1 = 3, u2 = 2
y_predicted = mynet(inputs]) 
Y_predicted = 2.9155 % the prediction given by the NN

Fine, the prediction is good.

But when I manually checked it by forward propagation, I got different result:

Z1 = W1*[3; 2] + b1; 
A1 = poslin(Z1); % applying ReLU activation function 
Z2 = W2*A1 + b2;
A2 = Z2; % linear activation function
y_predicted = A2;
y_predicted = 2.2549