How to fit data when yaxis in log scale and plot the fit?

Dear community please help me to solve the following problem:

I have a set of data plotted in Fig 1 with .YScale='log'. I select the data range and fit it and would like to plot the fir at the same figure 1. But, all the time when I plot the data i have warning "Warning: Negative data ignored".When I checked, I saw that fit generated the negative data...

I assume, I fit the data in some weared way which generates wrong a, b coeficients (y=-ax+b). The outcome , obwiously can not be plotted with log scale...

Please tell, how to solve this issue (how to fitt the data in log scale in fig 1)?!.

The part of the is below

Many thanks

Figure 1

[M,h]=contourf(xbin,ybin,f,cont_lev);
hold on
   ax=gca;
   ax.YScale='log';
   % ------ selecting the data region to fit 
    m=[];
    while true
        m=[m,M(:,2:M(2,1)+1)];
        M=M(:,M(2,1)+2:end);
        if isempty(M);
            break;
        end
    end

m=m';
m=sortrows(m);
idx= 10e-3>m(:,2) & m(:,2)>10e-6;

x=m(idx,1);
y=m(idx,2);
    %------------

polyfit(x,y,1)
x1=linspace(0,20,5)
y1=polyval(p,x1)
plot(x1,y1,'--','LineWidth',2)
hold off

[Mathieu NOE]

hello

so have you checked in first place that y does not contain zero and/ or negative values ?

[Sergii Snegir]

Dear Mathieu,

below is the outcome of the fit.... the fit generates somehow negative values. Will try to find a reason

[Mathieu NOE]

are you willing to share the code along with the data ?

[Sergii Snegir]

Here is the code and the data is ettached

clear all
opengl software

% working version V4_2. 




smth=1; % k-point mean values for smoothing the data
cont_lev=[.2 .2]; %contour level which is displayed
Imagformat='-dpng';   %format of exported  images  ('-djpeg' | '-dpng' | '-dtiff' | '-dpdf' | '-deps' | ...)

fit_my=["on"]; %implement this filter. If 'on' - applies fitting, 'off' - no fitting
fig_save=["off"];

timestamp =["20210201101947","20210208115740"];
%% -----


hold on
 leg_inf=[];
 for i=1:length(timestamp)
  % loading the data
    filePathName= sprintf('figures\\contours\\%s_from_2D-histo_counts.mat',timestamp(1,i));
    load (filePathName);
    f=counts';
    %----- filter low lewel signal of signal has less than 10% from Max
    %conts is is filtered
        max_val=max(f(:))
        repVal=max(f(:))*.1  % 0.1 = 10% from max value
        f(f(:,:)<repVal)=0;
        %
    
    
    f=normalize(f,'range',[0 1]);
    f=movmean(f,smth);
    max(f(:))
    tstapmt=timestamp(1,i);
    
    con_mat_siz=size(counts);
    xbin=xbin(1,1:con_mat_siz(1,1))';
    ybin=ybin(1,1:con_mat_siz(1,2))';
   
    
  % plotting contours
  
    plot_cont(f,xbin,ybin,i, cont_lev,tstapmt,smth,x_lim,y_lim,fit_my);
    
    
 end

hold off


%% functions

function plot_cont(f,xbin,ybin,i,cont_lev,tstapmt,smth,x_lim,y_lim,fit_my)
FonSize=14;
[M,h]=contourf(xbin,ybin,f,cont_lev);
% levels = get(h, 'LevelList')

ax = gca;
ax.XAxis.Label.String ="Bending (\mum)";
ax.YAxis.Label.String ="Conductance (G_0)";
ax.YAxis.TickValues =[1e-6 1e-5 1e-4 1e-3 1e-2 1e-1 1];
ax.LineWidth = 1;
ax.FontSize = FonSize;
ax.YScale='log';
ax.YGrid = 'on';
ax.XGrid = 'on';
ax.Layer = 'top';
ax.GridLineStyle = '-';
ax.GridColor='w';


ax.Color='#0039e6';
ax.LineWidth = 1;
ax.YMinorGrid = 'off';
    if i>1
    cont_col=["#ff0066","#00ff00","#ffcc00"];    
    h.LineColor=cont_col(1,i-1);
    h.Fill='off';
    h.LineWidth=1.5;
    end



ax.Box="on";
set(gcf,'PaperUnits','inches','PaperPosition',[0 0 8 6])

legend
ax.Legend.String(1,2*i-1)={tstapmt};
ax.Legend.Title.String = sprintf('con:%s smth:%s',string(cont_lev(1,1)),string(smth));
ax.Legend.Box='off';
ax.Legend.Color='white';
ax.Legend.TextColor='white';
ax.Legend.FontSize=12;

    if fit_my=='on'
        extr_fit_plot

    else
        ax.Legend.String(1,i)={tstapmt};
    end



% ------------function to extract the data from the counts for fitting sellect data set
    function extr_fit_plot
colFitLin=["#00ccff","#ff66ff","#00ff00"];
    m=[];
    while true
        m=[m,M(:,2:M(2,1)+1)];
        M=M(:,M(2,1)+2:end);
        if isempty(M);
            break;
        end
    end
m=m';
m=sortrows(m);
idx= 10e-3>m(:,2) & m(:,2)>10e-7;
%------------
x=m(idx,1);
y=m(idx,2);

p=polyfit(x,y,1)
x1=linspace(0,x_lim(1,2),5)
y1=polyval(p,x1)
plot(x1,y1,'--','LineWidth',2,"Color",colFitLin(1,i))
a=p(1,1);
b=p(1,2);

% info=sprintf('%.3f*x+%.3f\n',a,b);
info=sprintf('%d*x+%.d\n',a,b)
ax.Legend.String(1,2*i)={info};


end

end

Answer 1

[Mathieu NOE]

so this is my proposal - see the lines of your code I modified ; look for comments including :

% modified code

the original lines are still there but commented

one point still I coudn't fix completely, is how to have a better display of the fit function y = 10^(ax+b) in the legend

hope it helps !

clear all
opengl software
% working version V4_2. 
smth=1; % k-point mean values for smoothing the data
cont_lev=[.2 .2]; %contour level which is displayed
Imagformat='-dpng';   %format of exported  images  ('-djpeg' | '-dpng' | '-dtiff' | '-dpdf' | '-deps' | ...)
fit_my=["on"]; %implement this filter. If 'on' - applies fitting, 'off' - no fitting
fig_save=["off"];
timestamp =["20210201101947","20210208115740"];
%% -----
figure(1); % modified code

hold on
 leg_inf=[];
 for i=1:length(timestamp)
  % loading the data
%     filePathName= sprintf('figures\\contours\\%s_from_2D-histo_counts.mat',timestamp(1,i));
    filePathName= sprintf('%s_from_2D-histo_counts.mat',timestamp(1,i));
    load (filePathName);
    f=counts';
    %----- filter low lewel signal of signal has less than 10% from Max
    %conts is is filtered
        max_val=max(f(:))
        repVal=max(f(:))*.1  % 0.1 = 10% from max value
        f(f(:,:)<repVal)=0;
        %
    
    
    f=normalize(f,'range',[0 1]);
    f=movmean(f,smth);
    max(f(:))
    tstapmt=timestamp(1,i);
    
    con_mat_siz=size(counts);
    xbin=xbin(1,1:con_mat_siz(1,1))';
    ybin=ybin(1,1:con_mat_siz(1,2))';
   
    
  % plotting contours
    plot_cont(f,xbin,ybin,i, cont_lev,tstapmt,smth,x_lim,y_lim,fit_my);
       
 end
 
hold off

%% functions
function plot_cont(f,xbin,ybin,i,cont_lev,tstapmt,smth,x_lim,y_lim,fit_my)
FonSize=14;
[M,h]=contourf(xbin,ybin,f,cont_lev);
% levels = get(h, 'LevelList')
ax = gca;
ax.XAxis.Label.String ="Bending (\mum)";
ax.YAxis.Label.String ="Conductance (G_0)";
ax.YAxis.TickValues =[1e-6 1e-5 1e-4 1e-3 1e-2 1e-1 1];
ax.LineWidth = 1;
ax.FontSize = FonSize;
ax.YScale='log';
ax.YGrid = 'on';
ax.XGrid = 'on';
ax.Layer = 'top';
ax.GridLineStyle = '-';
ax.GridColor='w';
ax.Color='#0039e6';
ax.LineWidth = 1;
ax.YMinorGrid = 'off';
    if i>1
    cont_col=["#ff0066","#00ff00","#ffcc00"];    
    h.LineColor=cont_col(1,i-1);
    h.Fill='off';
    h.LineWidth=1.5;
    end
ax.Box="on";
set(gcf,'PaperUnits','inches','PaperPosition',[0 0 8 6])
legend
ax.Legend.String(1,2*i-1)={tstapmt};
ax.Legend.Title.String = sprintf('con:%s smth:%s',string(cont_lev(1,1)),string(smth));
ax.Legend.Box='off';
ax.Legend.Color='white';
ax.Legend.TextColor='white';
ax.Legend.FontSize=12;
    if fit_my=='on'
        extr_fit_plot
    else
        ax.Legend.String(1,i)={tstapmt};
    end
% ------------function to extract the data from the counts for fitting sellect data set
    function extr_fit_plot
colFitLin=["#00ccff","#ff66ff","#00ff00"];
    m=[];
    while true
        m=[m,M(:,2:M(2,1)+1)];
        M=M(:,M(2,1)+2:end);
        if isempty(M);
            break;
        end
    end
m=m';
m=sortrows(m);
idx= 10e-3>m(:,2) & m(:,2)>10e-7; 
%------------
x=m(idx,1);
y=m(idx,2);
% p=polyfit(x,y,1); % original code
p=polyfit(x,log10(y),1); % modified code / / fit polynomial on log of y
% x1=linspace(0,x_lim(1,2),5); % original code
x1=linspace(0,12,5); % modified code / / to not exceed x = 12 (my very own preference)
y1=polyval(p,x1);
y1= 10.^(y1); % modified code / / go back from log to lin y 

plot(x1,y1,'--','LineWidth',2,"Color",colFitLin(1,i))
a=p(1,1);
b=p(1,2);
% info=sprintf('%.3f*x+%.3f\n',a,b);
% info=sprintf('%d*x+%.d\n',a,b) % original code
if b <0 % do not put the '+' in the legend 
    info=sprintf('y = 10^^( %.3f*x %.3f )',a,b) % modified code / / remove return carriage
else % leave the '+' in the legend 
    info=sprintf('y = 10^^( %.3f*x+%.3f )',a,b) % modified code / / remove return carriage
end
ax.Legend.String(1,2*i)={info};
end
end

Answer 2

[Sergii Snegir]

Dear Methieu,

many, many thanks for your help. All works nice.

Only one question remaines. How to omit this warning in the code????

[Mathieu NOE]

Ok, I finally understood how to get the legend right

had to put the 'Interpreter' from 'text' (default) to 'none'

this is the new code :

clear all
opengl software
% working version V4_2. 
smth=1; % k-point mean values for smoothing the data
cont_lev=[.2 .2]; %contour level which is displayed
Imagformat='-dpng';   %format of exported  images  ('-djpeg' | '-dpng' | '-dtiff' | '-dpdf' | '-deps' | ...)
fit_my=["on"]; %implement this filter. If 'on' - applies fitting, 'off' - no fitting
fig_save=["off"];
timestamp =["20210201101947","20210208115740"];
%% -----
figure(1); % modified code

hold on
 leg_inf=[];
 for i=1:length(timestamp)
  % loading the data
%     filePathName= sprintf('figures\\contours\\%s_from_2D-histo_counts.mat',timestamp(1,i));
    filePathName= sprintf('%s_from_2D-histo_counts.mat',timestamp(1,i));
    load (filePathName);
    f=counts';
    %----- filter low lewel signal of signal has less than 10% from Max
    %conts is is filtered
        max_val=max(f(:))
        repVal=max(f(:))*.1  % 0.1 = 10% from max value
        f(f(:,:)<repVal)=0;
        %
    
    
    f=normalize(f,'range',[0 1]);
    f=movmean(f,smth);
    max(f(:))
    tstapmt=timestamp(1,i);
    
    con_mat_siz=size(counts);
    xbin=xbin(1,1:con_mat_siz(1,1))';
    ybin=ybin(1,1:con_mat_siz(1,2))';
   
    
  % plotting contours
    plot_cont(f,xbin,ybin,i, cont_lev,tstapmt,smth,x_lim,y_lim,fit_my);
       
 end
 
hold off

%% functions
function plot_cont(f,xbin,ybin,i,cont_lev,tstapmt,smth,x_lim,y_lim,fit_my)
FonSize=14;
[M,h]=contourf(xbin,ybin,f,cont_lev);
% levels = get(h, 'LevelList')
ax = gca;
ax.XAxis.Label.String ="Bending (\mum)";
ax.YAxis.Label.String ="Conductance (G_0)";
ax.YAxis.TickValues =[1e-6 1e-5 1e-4 1e-3 1e-2 1e-1 1];
ax.LineWidth = 1;
ax.FontSize = FonSize;
ax.YScale='log';
ax.YGrid = 'on';
ax.XGrid = 'on';
ax.Layer = 'top';
ax.GridLineStyle = '-';
ax.GridColor='w';
ax.Color='#0039e6';
ax.LineWidth = 1;
ax.YMinorGrid = 'off';
    if i>1
    cont_col=["#ff0066","#00ff00","#ffcc00"];    
    h.LineColor=cont_col(1,i-1);
    h.Fill='off';
    h.LineWidth=1.5;
    end
ax.Box="on";
set(gcf,'PaperUnits','inches','PaperPosition',[0 0 8 6])
legend
ax.Legend.String(1,2*i-1)={tstapmt};
ax.Legend.Title.String = sprintf('con:%s smth:%s',string(cont_lev(1,1)),string(smth));
ax.Legend.Box='off';
ax.Legend.Color='white';
ax.Legend.TextColor='white';
ax.Legend.FontSize=12;
    if fit_my=='on'
        extr_fit_plot
    else
        ax.Legend.String(1,i)={tstapmt};
    end
% ------------function to extract the data from the counts for fitting sellect data set
    function extr_fit_plot
colFitLin=["#00ccff","#ff66ff","#00ff00"];
    m=[];
    while true
        m=[m,M(:,2:M(2,1)+1)];
        M=M(:,M(2,1)+2:end);
        if isempty(M);
            break;
        end
    end
m=m';
m=sortrows(m);
idx= 10e-3>m(:,2) & m(:,2)>10e-7; 
%------------
x=m(idx,1);
y=m(idx,2);
% p=polyfit(x,y,1); % original code
p=polyfit(x,log10(y),1); % modified code / / fit polynomial on log of y
% x1=linspace(0,x_lim(1,2),5); % original code
x1=linspace(0,12,5); % modified code / / to not exceed x = 12 (my very own preference)
y1=polyval(p,x1);
y1= 10.^(y1); % modified code / / go back from log to lin y 

plot(x1,y1,'--','LineWidth',2,"Color",colFitLin(1,i))
a=p(1,1);
b=p(1,2);
% info=sprintf('%.3f*x+%.3f\n',a,b);
% info=sprintf('%d*x+%.d\n',a,b) % original code
if b <0 % do not put the '+' in the legend 
    info=sprintf('y = 10^( %.3f*x %.3f )',a,b) % modified code / / remove return carriage
else % leave the '+' in the legend 
    info=sprintf('y = 10^( %.3f*x+%.3f )',a,b) % modified code / / remove return carriage
end
ax.Legend.String(1,2*i)={info};
ax.Legend.Interpreter = 'none';
end
end