reverse engineering a simple neural network
7 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Hello, For teaching purposes, I am interested in using matlab to calculate the weights for a simple neural network classification problem, and then doing the classification myself using net1.IW, net1.LW, net1.b, etc. If I put the code below into a nn_bare.m file and run it, y_matlab and y_my are the same the first time, but y_my is then different the next time I run this. any hints for why the behavior is not consistent? I tried disabling adaptation, and adding a random number seed, but this did not help. Many thanks... E
%%data to be classified:
X=[2 7 0 7 4 3 9 1 8 4 10 6 4 5 2 3 5 8 3 4;
10 6 7 10 8 1 3 4 4 1 1 7 10 8 0 0 2 4 6 9];
y=[3 3 1 3 3 1 2 1 2 1 2 3 3 3 1 1 2 2 1 3];
%%for classification, turn labels into matrix format:
T=zeros(max(y),length(y)); for i=1:length(y); T(y(i),i)=1; end
rng('default'); % for reproducible results, as weights are initialized randomly
net1 = patternnet([5 5]);
net1.divideFcn=''; % don't divide data into training, testing, validation.
[net1,tr] = train(net1,X,T);
net1.adaptFcn=''; % don't change network during usage after training
%%use the trained network to classify a new point:
Xtest=[7 2]'
y_matlab=net1(Xtest)
%%Test matlab's classification manually:
a1 = tansig(net1.IW{1,:}*Xtest + net1.b{1});
a2 = tansig(net1.LW{2,1}*a1 + net1.b{2});
y_my = softmax(net1.LW{3,2}*a2 + net1.b{3});
y_my
Output: >> nn_bare
Xtest =
7
2
y_matlab =
0.0000
1.0000
0.0000
y_my =
0.0000
1.0000
0.0000
>> nn_bare
Xtest =
7
2
y_matlab =
0.0000
1.0000
0.0000
y_my =
0.0000
0.9307
0.0693
>>
0 commentaires
Réponse acceptée
Greg Heath
le 24 Avr 2016
You did not take into account the default normalization of inputs to the range [-1,1] and the unnormalization of the output from the range [-1,1].
Type, without the ending semicolons:
net = patternnet
inputprocessFcns = net.input.processFcns
outputprocessFcns = net.output.processFcns
Hope this helps.
Thank you for formally accepting my answer
Greg
3 commentaires
Greg Heath
le 25 Avr 2016
There is nothing keeping you from doing the normalizations/denormalization,
explicitly, before and after training.
Hope this helps.
Greg
Greg Heath
le 26 Avr 2016
I ran your code multiple times but always got the same result which differs from yours.
I also made comments that may be helpful.
GEH1 = ' Use lowercase for double variables (uppercase for cells)'
X=[2 7 0 7 4 3 9 1 8 4 10 6 4 5 2 3 5 8 3 4;
10 6 7 10 8 1 3 4 4 1 1 7 10 8 0 0 2 4 6 9];
y=[3 3 1 3 3 1 2 1 2 1 2 3 3 3 1 1 2 2 1 3];
GEH2 = ' [ I N ] = size(X) , [ O N ] = size(y)' % [2 20], [1 20 ]
% for classification, turn labels into matrix format: T = zeros(max(y),length(y)); for i=1:length(y); T(y(i),i)=1; end
GEH3 = 'Use functions ind2vec and, later, vec2ind'
rng('default'); % for reproducible results, as weights are initialized randomly net1 = patternnet([5 5]);
GEH4 = 'One hidden layer is sufficient for a universal approximator'
net1.divideFcn=''; % don't divide data into training, testing, validation.
GEH5 = ' Number of training equations Ntrneq = N*O = 20'
GEH6 = [ ' Number of unknown weights Nw = ( I + 1 )*H1 + ' ...,
'( H1 + 1 )*H2 + ( H2 + 1)*O = 15+30+6 = 51' ]
GEH7 = [ 'More unknowns(51) than equations (20) ==> ' ...
'OVERFITTING ==> ' ...
' 1. Reduce No. of hidden nodes or ' ...
' 2. use a validation set or' ...
' 3. use Bayesian Regularization ']
[net1,tr] = train(net1,X,T); net1.adaptFcn=''; % don't change network during usage after training % use the trained network to classify a new point:
GEH8 = 'Delete above command: it"s irrelevant'
Xtest=[7 2]'
Xtest =
7
2
GEH9 = ' Where is verification of good performance on training data ???'
y_matlab=net1(Xtest)
y_matlab =
2.7747e-07
1
3.5459e-07
% Test matlab's classification manually: a1 = tansig(net1.IW{1,:}*Xtest + net1.b{1}); a2 = tansig(net1.LW{2,1}*a1 + net1.b{2}); y_my = softmax(net1.LW{3,2}*a2 + net1.b{3});
y_my =
6.5673e-07
0.71821
0.28179
% Output: >> nn_bare % % Xtest = % 7 % 2 % % y_matlab = % 0.0000 % 1.0000 % 0.0000 % % y_my = % 0.0000 % 1.0000 % 0.0000 % % >> nn_bare % % Xtest = % 7 % 2 % % y_matlab = % 0.0000 % 1.0000 % 0.0000 % % y_my = % 0.0000 % 0.9307 % 0.0693
Not sure why my results are different from yours.
Hope this helps.
Greg
Plus de réponses (0)
Voir également
Catégories
En savoir plus sur Define Shallow Neural Network Architectures dans Help Center et File Exchange
Produits
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!