How to use a custom transfer function in neural net training

I want to use a function similar to tansig. I don't seem to be able to find a good example, and the tansig.apply method only allows me one line! I'm wrapped around this axle, and I suspect I'm missing something simple. Any ideas? I'm using 2012b.

Answer 1

[Greg Heath]

I cannot make sense of your post.

tanh(x) = tansig(x)

elliotsig(4*x) ~ tansig(x)

elliotsig4(x) ~ tansig(x)

function y = elliotsig4(x)

      y = x./(0.25 + abs(x));

end

What do you mean by the tansig.apply method ???

Sample code would help explain.

Hope this helps.

Thank you for formally accepting my answer.

Greg

[Greg Heath]

Note the following ranks in speed

1. tanh

2. elliotsig4

3. elliotsig

4. tansig

function elliottime
    clear all, clc
    tic; for i = 1:1e4
        y1 = tansig(1); % 0.7616
    end; t1 = toc % 1.2259
    
    tic; for i = 1:1e4
        y2 = elliotsig(4*1); % 0.8000
    end; t2= toc % 0.0029
    
    tic; for i = 1:1e4
        y3 = elliotsig4(1); % 0.8000
    end; t3 = toc % 0.0027
    
    tic; for i = 1:1e4
        y4 = tanh(1); % 0.7616
    end; t4 = toc % 5.94e-4
    
    [ [ t1 t2 t3 t4 ]' [y1 y2 y3 y4 ]' ]
    function y = elliotsig(x)
        y = x./(1+abs(x));
    end
    function y = elliotsig4(x)
        y = x./(0.25+abs(x));
    end
end

Answer 2

[Bob]

mladen,

The following has worked for me.

Move a copy of feedforwardnet.m into some directory on your path. Inside, you will find an assignment that sets the type of transfer to be used. Here's a snippet:

        net.layers{i}.size = param.hiddenSizes(i);
  % rcl
  %       net.layers{i}.transferFcn = 'tansig';
  %       net.layers{i}.transferFcn = 'satlins';
        net.layers{i}.transferFcn = 'bobfer';  

      end|

I invented a transfer called "bobfer". To implement it, go into .../toolbox/nnet/nnet/nntransfer.

Copy tansig.m and the folder +tansig to a directory on your path.

Give each a new name, such as bobfer.m and +bobfer.

Modify bobfer.m at the last line so it calls the bobfer.apply method:

a = bobfer.apply(n);

Now, in +bobfer, modify apply.m. This is pretty easy; it's only a few lines. This is where your transfer function goes.

Let me know if this helps...

Bob

[Nn Sagita]

Bob, I modified purelin transfer function, called 'mtf'. I saved in my working directory. I trained neural network and got outputs. But I got some messages too, like this:

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException at com.mathworks.toolbox.nnet.v6.diagram.nnTransfer.paint(nnTransfer.java:35) at com.mathworks.toolbox.nnet.v6.image.nnOffsetImage.paint(nnOffsetImage.java:49) at ....

Could you help me, what should I do?

[kelvina]

thanks bob, it helps me

but we can directly do this by coping file 'template transfer' from :C:\Program Files (x86)\MATLAB\R2010a\toolbox\nnet\nnet\nncustom

and just replace its function :a = apply_transfer(n,fp) by your function and then save this file in your working directory. it will work.

[Mayank Gupta]

Can you please explain in detail how to save a custom training function to the nntool directory ? I am using Firefly algorithm for optimization.

[Mehdi Jokar]

Bob, thank you for you instructions. but, is apply the only function that needs to be modified? or we need to modify the backprop and forwardprop function in the + folder ?

Mehdi

Answer 3

[Bob]

Greg, thanks for the answer. Let me improve my question.

I wish to use a customized transfer function, that is, I have a transfer function I want to use, and it is: x = y*(1 - (0.52*abs(y/2.6))) (for -2.5<y<2.5). The reason I wish to use this is that although tansig and elliotsig work nicely for me in simulation, I am putting the NN into an FPGA, and both tansig and elliotsig are difficult for me to compute in the FPGA. The equation above is easy to compute, and easy to combine with a saturation function.

I tried to follow the advice in the documentation, but had a heck of a time trying to modify tansig.m. Within tansig.m, there's a call to tansig.apply where the tansig function is actually computed, but I eventually figured out how to modify that.

So, my question was really: how do I use an arbitrary transfer function, not one of the existing functions.

Hope this qualifies my question. Sorry to send you off on a chase timing the various transfer functions.

Thanks, /Bob/

Answer 4

[Greg Heath]

I cannot understand why you think y2 is better than y1

x = -6:0.1:6;
y1 = x./(0.25+abs(x));
y2 = x.*(1 - (0.52*abs(x/2.6))) % (for -2.5<x<2.5).
figure
hold on
plot(x,y1)
plot(x,y2,'r')

Answer 5

[mladen]

Could anybody upload some examples of modified tansig.m and +tansig folder? This would be very helpful for my project and for other people too. Thank You.

[Nn Sagita]

If you have some examples how to modify transfer function, please share for me. Thank you.

Answer 6

[mladen]

Thank you Bob. Nice trick with feedforwardnet.m (good for permanent use). I've managed to do this but some new questions arise:

1. How to use param in apply(n,param) ? (more info-> matlabcentral/answers/686)?
2. How to use different transfer functions within the same layer?
3. My apply function looks something like this:

function A = apply(n,param)
%....
A=a1.*a2;
end

now I would like to use a1 and a2 to speedup the derivative computation in da_dn.m (this has already been done with tansig.m, but with the final value (A in my code))...is it possible?