Bootstrap for surface fit goodness of fit stats

I am trying to use bootstrap on the MSE, R-squared output generated from a surface fit. However, I am having trouble figuring out how to set it up with the correct arguments.

I tried to do something like:

[fitresult1000,gof1000] = bootstrp(1000,'fit',x,y,z,'poly22',opts);

but I always get the following error message

Unable to evaluate BOOTFUN with the supplied arguments.

I figure this should be pretty simple, so I don't know what I am missing, if I can't figure this out I will probably just hand code it in Python, however it seems like a waste of time if Matlab has a simple function that will do it.

Anyway, any input you could give me would be appreciated.

Answer 1

[Richard Willey]

Few quick observations

First: There are a lot of different ways to bootstrap a regression model. Some of the more common examples include

Parametric residual bootstrap:

• Start by generating a regression model
• Calculate the variance of the residuals
• Using this information to generate new residuals
• Generate new observations by adding new residuals to YHat

Nonparametric residual bootstrap

• Start by generating a regression model
• Calculate the residuals
• Generate new observations by sampling with replacement from the set of residuals and adding these to YHat

Paired bootstrap

• Sample with replacement from the set of observations

Each of these techniques has its own advantages and disadvantages. (At the most basic level, there is a tradeoff between convergence speed and how restrictive your assumptions are about the data set) For basic information about bootstrapping regression models, I strongly recommend the following

http://www.economics.uci.edu/~dbrownst/bootmi.pdf

Second: There is also the whole issue about how best to get this up and running in MATLAB

Here’s some simple code that will show you how to bootstrap SSE and R^2 using a parametric residual bootstrap. You can easily change this to a nonparametric residual bootstrap using the randsample function to sample with replacement.

X1 = 100 * rand(100,1);
X2 = 100 * rand(100,1);
Y = 2*X1 + 3*X2 + 10*randn(100,1);

[foo GoF] = fit([X1 X2],Y, 'poly11')

YHat = foo(X1,X2);
resid = Y - YHat;

Scaling = std(resid);

% Simple bootstrap example

N_Boot = 1000;

SSE = zeros(N_Boot,1);
R_Sqrd = zeros(N_Boot,1);

for i = 1:N_Boot

     [foo_b , GoF_b] = fit([X1 X2],YHat + Scaling * randn(100,1), 'poly11');

     SSE(i) = GoF_b.sse;
     R_Sqrd(i) = GoF_b.rsquare;

end

mean(SSE)
std(SSE)
mean(R_Sqrd)
std(R_Sqrd)

Answer 2

[Walter Roberson]

The documentation for bootstrp indicates (at least in the current version) that,

bootfun is a function handle specified with @.

You have instead passed a string.

Answer 3

[Juan Leon]

I still get the same error message when I use @fit instead of 'fit'.

I guess I am not sure how I am supposed to pass in the BOOTFUN which in this case is:

fit([x,y,], z,'poly22',opts);

from the documentation, the input arguments passed into bootstrp are scalars, column vectors, or matrices, so I don't know how the 'poly22' and opts would be passed in.

Answer 4

[Drew Steen]

Hard to tell without knowing the details of the function 'fit' you're using, but I think the issue is that bootstrp will only accept numerical data as arguements. I believe the string 'poly22' is throwing it off.