Non-Linear Regression Methods

Hi everyone, currently I want to study different regression methods with the help of MATLAB built-in function and compare them according to how well the methods fit. Basically, I have got a set of Data which give me 3D cures (x,y,z) when I use lsqcurvefit and it works pretty well. I have also look into different methods for non-linear regressions such as fitnlm, glmfit and lsqnonlin (same as lsqcurvefit?). However in fitnlm, the input format can only be

nlm = fitnlm(x,y,@function,beta0)

like most of them do not have same format of input as lsqcurvefit allowing me to input x,y,z in the function. Therefore, can I have some suggestions which allow me to fit 3D nonlinear data other than the above options? And how do I know how well will the function fits? Or did I use fitnlm glmfit wrongly?

Answer 1

[John D'Errico]

READ THE HELP.

For example, a fragment of the help for fitnlm:

NLM = fitnlm(X,Y,MODELFUN,BETA0) fits a nonlinear regression model
    using the column vector Y as a response variable and the columns of the
    matrix X as predictor variables.

So if you have TWO independent variables, they would be passed in as two columns of the ARRAY X.

As far as LSQNONLIN goes, it can handle any number of independent variables, because it only cares about the residual vector. You don't even pass in data to LSQNONLIN, just a function.

GLMFIT does not accept nonlinear models. But as long as the model is linear in the coefficients, there is no problem. It can still be nonlinear in the independent variable, since you pass in an array of data. And it does handle the case of multiple independent variables, since you must create the array X.

Of course, there is also NLINFIT, which is able to handle an array of multiple independent variables.

So basically, you need to read the help. Different tools handle a variety of problems.

[Ho Nam Ernest Yim]

Sorry, I don't get what is the response variable and predictor variables. So, if I have now got x y z as in all single column vectors. Will they my response variable? And predictor variables will be?

[Ho Nam Ernest Yim]

If my x y z is cure, temperature and cure rate. So cure is the output; temperature (unchanged) and cure rate is depended on cure. Am I correct, if x is my predictor and y, z will be my response variable? So my input should be

    nlm = fitnlm(x,[temperature cure rate],@function, initial guess  of input)

[John D'Errico]

You have not been clear about your problem.

Assuming that you are trying to model a surface, thus z(x,y)

X = [x(:),y(:)]
Y = z(:);

x and y are called independent variables, or predictor variables. z is the dependent variable, thus the response variable.

The idea is you can try to evaluate this unknown function for any combination of x and y, getting out z(x,y).

This presumes that the noise lies in z, as a "measured" value. It is the classic problem that tools like lsqcurvefit, lsqnonlin, fitglm, nlinfit, are all designed to solve.

Your questions seem to imply that your problem is not a classic one. But you have not said enough to know what it is that makes it non-classic.

If your problem has noise in each of x, y, and z, then this is called an errors in variables problems, or sometimes total least squares. It is much more difficult to handle. None of the codes that you mention handle that problem. Use of a standard tool that presumes error only in z will produce results with biased estimates of the parameters, so while it will provide a solution, the solution will be problematic under conditions of significant noise or large residuals.

Note that lack of fit of your model to the data will be interpreted as noise. Even though lack of fit will usually be seen as patterned noise, it is still residual error. Any such tool assume the model that you pose is the correct model, so any residuals must be in the form of "noise".

If your problem is not a surface, but some general curvilinear path through the 3-d space (x,y,z), then again, this is a case where you will have problems. Now, people seem to think they can just try to solve for the coefficients using nonlinear regression. In fact, this is again an errors in variables problem, because the equation of the path is usually an implicit function of all three variables.

So without more clarity on your part about the problem you may want to solve, I cannot say much more without writing at least several books on the subject.

[Ho Nam Ernest Yim]

Thank you so much!! I am more clear to my 'predictor variables' and 'dependent variable'. I have now tried to use this functions and they work.

   output = nlinfit(x, z, @function, beta0);

However, when I input:

output = fitnlm(x,z,@function,beta0)

It shows :

Error using internal.stats.getscheffeparam>ValidateParameters (line 182)
If non-empty, JW must be a numeric, real matrix.

Error in internal.stats.getscheffeparam (line 110)
    [J,VF,VP,JW,Intopt,TolSVD,TolE,VQ,usingJ] = ValidateParameters(J,VF,VP,JW,Intopt,TolSVD,TolE,VQ,allowedIntopt);

Error in nlinfit (line 433)
        sch = internal.stats.getscheffeparam('WeightedJacobian',J(~nans,:),'Intopt','observation','VQ',VQ);

Error in NonLinearModel/fitter (line 1123)
                    nlinfit(X,y,F,b0,opts,wtargs{:},errormodelargs{:});

Error in classreg.regr.FitObject/doFit (line 94)
            model = fitter(model);

Error in NonLinearModel.fit (line 1430)
            model = doFit(model);

Error in fitnlm (line 94)
model = NonLinearModel.fit(X,varargin{:});

I am currently thinking it's because my x is not a 'n-by-p matrix', am I correct?