sbiopredictionci return empty result

Hello team,

I was trying to comput the prediction confident interval for my parameter estimation results.

I first used sbioparameterci to check with my estimated parameters.

The estimattions are not perfect, but all of the status for my estimated parameters were either estimatable or constrained.

Thus, I further use sbiopredictionci to get the prediction confident interval.

By excuting the following code for defults setting, I was able to get prediction CI, but the results were not ideal.

ciPred = sbiopredictionci(fitWeightFunction);

Thus, I tried to use bootstrap method with following code to see if I could get better results.

ciPred = sbiopredictionci(fitWeightFunction, 'Alpha', 0.1, 'Type', 'bootstrap', 'NumSamples', 1000, 'UseParallel', true,'Display','final')

However, the ciPred I got is an empty output '0x0 PredictionConfidenceInterval'

Please find the OptimResults of my fitWeightFunction data in the attachment file.

Thank you very much.

Best,

Jesse

Answer 1

[Jeremy Huard]

Hi Jesse,

it looks like your model uses an external function calculate_P_A2B_factor. You will need to pass this function to the parallel pool you are using.

So, if you already have an open parallel pool, you can enter the following command before calling sbiopredictionci:

addAttachedFiles(gcp,"calculate_P_A2B_factor.m")

If your parallel pool is not open yet, you can open it and attach the file with this command:

c = parcluster('local');
parpool('local',c.NumWorkers,AttachedFiles="calculate_P_A2B_factor.m");

Best regards,

Jérémy

[Jesse Chao]

Hello Jérémy,

Thank you for saving my day!

However, how could I know it is caused by this external function?

Is there anyway I could find this inforamtion?

I will be super helpful since I have several external function embedded.

Thank you very much.

Best,

Jesse

[Jeremy Huard]

Hi Jesse,

This information shows up as a warning when you try and run sbiopredictionci.

In R2022b I got the message:

Warning: One or more simulations failed because required functions could not be found. If you are simulating in parallel with the UseParallel
option, you may need to send files to the workers using the addAttachedFiles command. The following error occurred:
Undefined function 'calculate_P_A2B_factor' for input arguments of type 'double'. 

If you know which functions the model needs to run, you can pass the list directly to addAttachedFiles:

addAttachedFiles(gcp,["file1.m", "file2.m"]);

Also please give the file name with its extension. I forgot it in my previous post that I have now corrected.

Best,

Jérémy

[Jesse Chao]

Hello Jérémy,

Thank you so so much!

I forgot that I usually turn off all the warnings when I ran my simulation and fitting in simbiology...

so I didn't see this information.

Thank you again.

Have a ncie day.

Best,

Jesse

[Jesse Chao]

Hello Jérémy,

I just tried to do the same thing with adding the line you provided

addAttachedFiles(gcp,"calculate_P_A2B_factor.m")

However, I got the following warning, and this code ran forever unstop even with only 3 smaples...

ciPred = sbiopredictionci(fitWeightFunction, 'Alpha', 0.1, 'Type', 'bootstrap', 'NumSamples', 3, 'UseParallel', true,'Display','final');
Warning: Observation data contains NaN. This may cause the computation of 'bootstrap' confidence intervals to error. 
> In SimBiology.fit/LeastSquaresResults/validateDataForBootstrapping (line 2685)
In SimBiology.fit/LeastSquaresResults/validatePredictionCIInput (line 2633)
In SimBiology.fit/LeastSquaresResults/predictionci (line 1052)
In sbiopredictionci (line 68) 

Computing 90% bootstrap prediction confidence intervals
-------------------------------------------------------
Warning: You have passed PARTICLESWARM options to LSQNONLIN. LSQNONLIN will use the common options and ignore the PARTICLESWARM options that do not apply.
To avoid this warning, convert the PARTICLESWARM options using OPTIMOPTIONS. 
> In optim.options/SolverOptions/convertForSolver (line 777)
In SimBiology.fit.internal.validateOptions>convertOptionsForMethod (line 167)
In SimBiology.fit.internal.validateOptions (line 15)
In SimBiology.fit.internal/FitObject/initializeReRun (line 376)
In SimBiology.fit.internal/FitObject/rerunFit (line 314)
In SimBiology.fit.LeastSquaresResults.createBootFun/getBootstrapSample (line 1806)
In SimBiology.fit.LeastSquaresResults>@(x)getBootstrapSample(x,numGroups,groupIdx,groupSize,resampleWeights,refIdx,obj.Data,obj.Weights,obj.ConfidenceIntervalData.EstimInfo,restartData,obj.Beta.Estimate,exitFlag,fcn) (line 1764)
In bootci (line 146)
In SimBiology.fit/LeastSquaresResults/predCIBootstrap (line 1576)
In SimBiology.fit.LeastSquaresResults>@(obj)predCIBootstrap(obj,alpha,numSamples,useParallel) (line 1060)
In SimBiology.fit/LeastSquaresResults/dispatchConfidenceIntervalComputation (line 1880)
In SimBiology.fit/LeastSquaresResults/predictionci (line 1068)
In sbiopredictionci (line 68) 
Warning: You have passed PARTICLESWARM options to LSQNONLIN. LSQNONLIN will use the common options and ignore the PARTICLESWARM options that do not apply.
To avoid this warning, convert the PARTICLESWARM options using OPTIMOPTIONS. 
> In optim.options/SolverOptions/convertForSolver (line 777)
In SimBiology.fit.internal.validateOptions>convertOptionsForMethod (line 167)
In SimBiology.fit.internal.validateOptions (line 15)
In SimBiology.fit.internal/FitObject/initializeReRun (line 376)
In SimBiology.fit.internal/FitObject/rerunFit (line 314)
In SimBiology.fit.LeastSquaresResults.createBootFun/getBootstrapSample (line 1806)
In SimBiology.fit.LeastSquaresResults>@(x)getBootstrapSample(x,numGroups,groupIdx,groupSize,resampleWeights,refIdx,obj.Data,obj.Weights,obj.ConfidenceIntervalData.EstimInfo,restartData,obj.Beta.Estimate,exitFlag,fcn) (line 1764)
In bootstrp (line 178)
In bootci>bootper (line 207)
In bootci (line 167)
In SimBiology.fit/LeastSquaresResults/predCIBootstrap (line 1576)
In SimBiology.fit.LeastSquaresResults>@(obj)predCIBootstrap(obj,alpha,numSamples,useParallel) (line 1060)
In SimBiology.fit/LeastSquaresResults/dispatchConfidenceIntervalComputation (line 1880)
In SimBiology.fit/LeastSquaresResults/predictionci (line 1068)
In sbiopredictionci (line 68) 

Is there anyway I could resolve it?

Please let me know. Thank you in advance.

Best,

Jesse

[Jeremy Huard]

Hi Jesse,

sorry, I had missed your reply.

I can't say much without the fit object passed to sbiopredictionci. But from the warning I suspect the options passed to sbiofit wwere created with optimoptions('particleswarm') but you called sbiofit with lsqnonlin as optimizer. Could it be?

Would you be able to share fitWeightFunction in a MAT via private message?

If not, you could try to run sbiofit with lsqnonlin before calling sbiopredictionci to check that it completes in a reasonable amount of time.

Best,

Jérémy

[Jesse Chao]

Hello Jérémy,

Indeed, I did double chack on the optimoptions('particleswarm'). It is a bug in my script.

However, after I corrected this mistake with the correct lsqnonlin, the problem remained.

I have attached the corrected version of the fitWeightFunction considering the optimoption error with the name of fitComb in the attachment file in this post.

Hope you could provide me with some insights. Thank you very much.

Best,

Jesse

[Jeremy Huard]

Hi Jesse,

could you please send the calculate_P_A2B_factor.m file as well as the script you use to call sbiopredictionci?

Thanks,

Jérémy