MATLAB Answers

Neural Network - How to use a custom performance function

116 views (last 30 days)
C
C on 20 Feb 2013
Commented: Gokhan Demirkiran on 10 Feb 2017
I would like to train a network using the cross entropy error function. I am using the Neural Network Toolbox version 7.0.1.
My understanding is that it is not available as a built in option (via property net.performFcn), as stated here:
However that documentation says that I can write my own custom performance function. However, I can not find documentation for doing this. The section referenced, the chapter on custom networks, does not have this, as seen here (the example there uses the built in mse performance function)
I did find another section of the documentation titled "Custom Functions",
This mentions template functions, which are apparently m files meant to be copied and customized for use in making custom functions. However, typing help nncustom as shown in the documentation actually returns "nncustom not found".
I have since resorted to editing the mse performance function directly (mse.m), which upon renaming apparently can be used via net.performFcn='myfun'.
The code within this m file is hard to understand. I believe it should return the value of the cost function, and it should return the gradient at a given point. Can someone tell me where these two things are within this m file? The output arguments, out1 and out2, aren't even described in the function's help text. Am I right in guessing that these two outputs are the cost function value and gradient?
Thanks in advance.

  2 Comments

Sign in to comment.

More Answers (3)

Andrew Duncan
Andrew Duncan on 15 Aug 2013
Apparently the technique is to overload your own function in place of mse (or one of the other 3 standard functions). It sounds like you are doing this correctly, "C", but you comment:
"The code within this m file is hard to understand. I believe it should return the value of the cost function, and it should return the gradient at a given point. Can someone tell me where these two things are within this m file? The output arguments, out1 and out2, aren't even described in the function's help text. Am I right in guessing that these two outputs are the cost function value and gradient?"
I don't think you should be trying to modify the function call definition itself. This is all part of the standard boilerplate code for argument passing into / out of the function. It is likely to be safe to ignore this boilerplate code (leave well alone!) and instead work on the definitions described by Amaria in the % commented out code below... What will come out of out1 and out2 will (I think) be defined by the .trainParam and/or .performParam settings you apply. The mean squared error function itself is implemented in this sub-function of mse.m (mymse.m):
function [perfy,n] = performance_y(net,t,y,ew,param)
e = gsubtract(t,y);
e = adjust_error(net,e,ew,param);
[perfy,n] = meansqr(e);
perfy = perfy * (1-param.regularization);
end
...so I am guessing you need to replace meansqr(e) with your own function? You may also need to modify:
function d = dperf_dy(net,t,y,ew,perf,param)
...to take care of the gradient descent?
Here is an example followed by instructions on the (sub)functions (within mymse.m) that you may want to modify...
%%%BEGIN CODE%%%
% Thanks to Amaria Zidouk, Technical Support Team, MathWorks for this...
% Make a copy of mse function in your current working directory
% (Note: mae, sse, sae could also be used as starting points)
copyfile([matlabroot '\toolbox\nnet\nnet\nnperformance\mse.m'], 'mymse.m');
% load a simple example input and target dataset
[x,t] = simplefit_dataset;
% Create an example fitting ANN with 10 hidden units and using the scaled
% conjugate gradients training function
net = fitnet(10,'trainscg');
% Change the performance function (fitness function) to our local copy
% Note!! The name of the function inside mymse.m *MUST* be kept as 'mse'
% as in:
% function [out1,out2] = mse(varargin)
% This ensures our user defined function is overloaded in place of the
% standard function 'mse' and that the error checking inside NN toolbox
% does not throw up an error!
net.performFcn = 'mymse';
% Prove that our new (so far identical) mymse.m function can be used
% correctly and works.
net = train(net,x,t);
% You are now free to modify mymse.m to suit your new performance/fitness
% function requirements
%%%END CODE%%%
Note: the bits you will probably need to modify start below the BOILERPLATE_END marker around line 518 in version 2012a. The bits above that mainly deal with argument handling and probably need to be left as standard.
Also Amaria has added the following notes:
The best way to create a custom performance function is to use MSE.M and the +MSE package of functions as a template. [i.e. as above] Here are descriptions of the required package functions:
newfcn.m - Same as mse.m
+newfcn/apply.m - The main performance calculation
function perfs = apply(t,y,e,param)
% Calculate performance for each target individually so perfs is same size
% as t, y and e.
+newfcn/backprop.m - Backprop derivatives
function dy = backprop(t,y,e,param)
% Return dperf/dy, the derivative of performance with respect to each
% output y.
+newfcn/forwardprop.m - Forward propagate derivatives
function dperf = forwardprop(dy,t,y,e,param) Return dperf/dwb given
% dy/dwb.
+newfcn/type.m - Indicate that "newfcn" is a performance function.
function t = type
% t = 'performance_fcn';
+newfcn/name.m
function name = name()
% Return the name of the function, for instance: name =
% 'My NewPeformance Function';
+newfcn/normalize.m
function flag = normalize
% Return true if mean performance is desired (i.e. mean squared error,
% mean absolute error) and false if the absolute performance is desired
% (i.e. sum squared error, sum absolute error).
+newfcn/parameterInfo.m
function param = parameterInfo
% Return the same array of parameter definitions as MSE. Customer can
% also add additional parameters if desired.
+newfcn/perfwb.m - Regularization performance used to minimize weights
% +and biases
function perf = perfwb(wb,param)
% Return the performance measure for weights and biases. This performance
% measure is only used when net.performParam.regularization is set to a
% value greater than 0. If regularization is not going to be used this
% function can return 0.
+newfcn/dperf_dwb.m - Regularization derivatives
function dperf = dperf_dwb(wb,param)
% If you are not doing regularization then this function can return zeros
% the same size as wb. Otherwise it should return a derivative of
% regularization performance (calculated by perfwb) with respect to each
% weight and bias value in wb.
% Please note that there is one more caveat when following this approach
% in R2012b – there is an issue that is planned to be resolved in a future
% release, but currently defining custom functions with this approach
% works only with the non-MEX version of the Neural Network code, so it is
% necessary to call TRAIN with a special syntax – i.e., using the nn7
% option. This workaround is the following syntax for training and
% simulating the network:
%%%BEGIN CODE%%%
net = train(net,x,t,nn7);
y = net(x,nn7);
%%%END CODE%%%
% The problem is that a check for custom performance functions fails with
% the new default MEX implementations. The MEX calculations do not
% support custom functions yet, so calculations are supposed to
% revert to MATLAB.
% We plan to address this for a future release but the workaround in the
% meantime is to include nn7 as the last argument for TRAIN or simulating
% a network.
% Additionally, the function files and packages to use for defining custom
% versions of other Neural Network components can be found by executing
% the following in MATLAB:
%%%BEGIN CODE%%%
doc nncustom
%%%END CODE%%%
I hope this helps?

  8 Comments

Show 5 older comments
Giorgio
Giorgio on 1 Apr 2015
Hi Vincent, mymse function does not work despite it is the copy of the original mse function and i don't get it working.....I did the following: 1) I copied the mse.m and +mse folder from inside the matlab directory to my working directory and i renamed them as mymse.m and +mymse. 2) I modified the function name inside the mymse.m file from:
function perf = mse(net,varargin)
to
function perf = mymse(net,varargin)
then building the net, i added my own performance function as
net.performFcn = 'mymse'
3) I left the .m functions inside the +mymse folder as they are since mymse function computes the mse.
It should be straightforward but it doesn't at all !!!
Why doesn't it work ??? I don't get it!!! always a flatline in the resulting model.....
Maybe Greg Heath can help us.
Thanks.
Dominykas Mostauskis
Dominykas Mostauskis on 15 Feb 2016
It's not immediately clear that the first suggestion of copying the `mse.m` file and overloading the vanilla function is meant for older releases (pre-2012?). Newer releases should use the latter method. Here is the question specifically for post-2012 releases.
Gokhan Demirkiran
Gokhan Demirkiran on 10 Feb 2017
you may have to replace train(net,x,t) with train(net,x,t,nn7)

Sign in to comment.


Greg Heath
Greg Heath on 12 Feb 2014

  1 Comment

Greg Heath
Greg Heath on 6 Jan 2015
Oh how times have changed!
I converted mse to xent back in 2004. I think I had to modify one line of internal code.
I guess I was standing in the way of progress (:>)

Sign in to comment.


Scott Otterson
Scott Otterson on 8 Jun 2013
I agree that there is little documentation on this topic.
But here's a link to a page that at least provides a link to a code template (performance1.m): http://www.mathworks.de/support/solutions/en/data/1-2IE8FD/index.html?product=SL&solution=1-2IE8FD

  0 Comments

Sign in to comment.

Sign in to answer this question.