How to get cross validation (crossval) to work with output from fitted generalized linear model (fitglm).

X is an n x k matrix of predictors. Y is n x 1 vector of responses; this is binary. I implement a probit, generate predictions from the probit, and then assign each response a 1 or zero depending on whether it is greater than 1/2.

I am using cross validation to get an estimate of the model's classification error, not to choose the Xs. In particular, I run

fcn = @(Xtr,Ytr,Xte) (predict( fitglm(Xtr,Ytr,'Distribution','binomial','Link','probit'), Xte) > 0.5);

rng(1979);

mcr = crossval('mcr',X,Y,'Predfun',fcn,'kfold',5);

When I run this, Matlab always gives me mcr=1. Using subsets of the data, I've estimated the model, made predictions on the left out subset, and then compared those to the true responses (for that left out subset). The model consistently gets about 65-70% of the predictions correct.

Any help is much appreciated.

Answer 1

[Chaitral Date]

Will it be possible for you to send us the input data(X,Y)? Send us the complete reproduction steps and then we can investigate upon this.