Problem with "Reshape"

Hi! This might be a hard question to explain, but will do my best.

We constantly get this error message:

Error using reshape
To RESHAPE the number of elements must not change.

Error in DREAM_initialize

Error in DREAM

Error in runDREAM_4 (line 1231)
[chain,output,fx] = DREAM(Func_name,DREAMPar,Par_info,Meas_info,options);

The script "Dream" calibrates parameters for another function. We have tried to transpose our data so it should fit, but this does not help.

Our code look like this:

if example == 20, % Fit the van Genuchten solution of the contaminant transport equation to measured data (Derived from Example 4)
% Problem specific parameter settings
DREAMPar.d = 3; % Dimension of the problem
DREAMPar.N = 10; % Number of Markov chains
DREAMPar.T = 500; % Number of generations
DREAMPar.lik = 11; % Gaussian likelihood with measurement integrated out (see file
% Calc_density.m and Table 2, p. 29 of DREAM manual for reference)
options.modout = 'yes'; % Return model (function) simulations of samples Yes or No)?
% Define modelName
Func_name = 'WaterqualityStr4_fit';
% Give the parameter ranges (minimum and maximum values). [v DD tp]
Par_info.min = [10 0 0 1 0 0 1];
Par_info.max = [60 2e5 5e5 1e7 2e5 5e5 1e7];
Par_info.initial = 'latin'
Par_info.boundhandling = 'reflect'; % Explicit boundary handling

% Load the experimental data
load dataAssignment_04.mat;
fieldnames(dataChassieu);
global file
file = dataChassieu;

% Define constant input parameters
global Extra
%Extra.Q=[(dataChassieu(1).flow)';(dataChassieu(2).flow)';(dataChassieu(3).flow)';(dataChassieu(4).flow)';(dataChassieu(5).flow)';(dataChassieu(6).flow)';(dataChassieu(7).flow)';(dataChassieu(8).flow)';(dataChassieu(9).flow)'];
Extra.Q=(dataChassieu(1).flow)'
%Extra.q=q;
Extra.t=(dataChassieu(1).time)';
Extra.distPuls_1=2e5; % distPuls_1
Extra.injMass_1=5e5; % injMass_1
Extra.D_1=1e7; % D_1
Extra.distPuls_2=2e5;% distPuls_2
Extra.injMass_2=5e5; % injMass_1
Extra.D_2=1e7; % D_1

% Define the measured concentrations
%Meas_info.Y=[(dataChassieu(1).TSS)';(dataChassieu(2).TSS)';(dataChassieu(3).TSS)';(dataChassieu(4).TSS)';(dataChassieu(5).TSS)';(dataChassieu(6).TSS)';(dataChassieu(7).TSS)';(dataChassieu(8).TSS)';(dataChassieu(9).TSS)'];
Meas_info.Y=dataChassieu(1).TSS';
%Extra.flow = dataChassieu(1).flow; 
% Run the DREAM algorithm
[chain,output,fx] = DREAM(Func_name,DREAMPar,Par_info,Meas_info,options);
% Post-processing
addpath('PostProcessing')
DREAM_postproc
% Modify labels  Note these lines would be better placed in the file postprocMCMC.m
% but are included here to make things easier for the students.
figure(10) % select figure 10
xlabel('time [s]','fontsize',14,'fontweight','bold','fontname','bold','fontname','times');
ylabel('concentration [mg/L]','fontsize',14,'fontweight','bold','fontname','bold','fontname','times');
legend('95% confidence bounds (total uncertainity)','analytical solution (parameter uncertainty)','experimental data');
close(11); %close graph on residual autocorrelation (not needed for course)
close(12); %close graph on QQ posterior vs normal distribution (not needed for the course)
end

% Create a single matrix with values sampled by chains
ParSet = GenParSet(chain);

The function, "WaterqualityStr4_fit" looks like this:

function cvG = WaterqualityStr4_fit(Par,file,Extra)
global file
global Extra

%All par values have been changed from Pars to Par
% parameters to be fitted - based on the results from Morris we the most
% important parameters where 1. TSS_back, 2. injMass_2 and 3. D_2
TSS_back=Par(1); %TSS_back - background concentration
distPuls_1=Par(2);
injMass_1=Par(3);
D_1=Par(4);
distPuls_2=Par(5);
injMass_2=Par(6);% injMass_2
D_2=Par(7);% D_2

% Other function parameters passed through the Extra global variable

global Extra
Extra.dt=1;
Extra.flow = file(1).flow; 
% distPuls_1=Extra.distPuls_1; % distPuls_1
% injMass_1=Extra.injMass_1; % injMass_1
% D_1=Extra.D_1; % D_1
% distPuls_2=Extra.distPuls_2;% distPuls_2
% 

%%inserted from WaterqualityStr4

%%define additional data and information
%Q=Extra.flow; % measured flow [m3/s]
t=(1:length(Extra.Q))*Extra.dt*60; % time vector [min]->[s]
num_pulses=(length(Par)-1)/3; % the number of injections in the sewer system, it is a predefined number.

%%define function parameters
TSS_back=Par(1); % [g/m3] baseline TSS concentration

% inizialize parameter vectors
distPuls =nan(num_pulses,1); % [m] distance from TSS point of injection
injMass=nan(num_pulses,1); % [g] TSS mass injected
dispCoef=nan(num_pulses,1); % [m2/s] dispersion coefficient
for i=1:num_pulses
    distPuls(i)=Par(1+i);
    injMass(i)=Par(1+num_pulses+i);
    dispCoef(i)=Par(1+2*num_pulses+i);
end

%%Hydraulic variables (used for transport)
% calculate water level based on Q-h relationship
h=0.0146883323.*((Extra.Q*1e3).^0.4678812104) ; % [m] wather depth, regression for Chassieu (calculated using Q in l/s)
D=1.6; % [m] diameter fo the pipeline

theta=acos(1-2*mean(h)./D); % [radians] angle of wet area (average)
A=(D^2)/4*(theta-sin(2*theta)/2); % [m2] wet area
u=nanmean(Extra.Q/A); % [m/s] mean velocity

%%calculate TSS transport (based on diffusion-advection equation in one dimensional flows
TSSconc=zeros(num_pulses,length(Extra.Q)); % a TSS profile for each injection

for i=1:num_pulses
    for j=1:length(t)
       TSSconc(i,j)=injMass(i)/(4*A*pi()*dispCoef(i)*t(j))^.5 * ...
            exp(-(distPuls(i)-u*t(j))^2/(4*dispCoef(i)*t(j)));
    end
end

%%overlap TSS curves and add background concentrations

   cvG=sum(TSSconc,1)+TSS_back;

end

Sorry for the long post! I hope somebody is able to help us out! Can anybody figure out what we are doing wrong?

Kind Regards Ida

[Rik]

Why are you using globals? They make for programs that are hard to debug. You almost never need them. I have only needed them ever once, to preserve compatibility with an old release where I could use guidata reliably.

If you absolutely need to use a global, at least give it a name you can be sure of to never ever use in another script to prevent unwanted changes.

A more general tip: try to reduce your code to a small example we can copy, paste and reproduce. Often this will help you find the error yourself. If it doesn't, you've made a MWE (or MFE, minimal failing example, if you're being pedantic), which makes it easier for other to find out what is going wrong. (also, I can't find a reshape in your code)

[Adam]

Just use the debugger! It is trivial to look at the sizes of the components given to a function if you use breakpoints. Stop (or Pause in R2017b) on errors together with some quick command line tests and common sense will locate the problem faster than it takes you to put your question in the forum.

Also don't forget to read the actual error message. Transposing a vector does nothing to change the number of elements so clearly this is not going to solve a problem where your call to reshape is changing the number of elements.

The error points to RunDream_4 which is not included anywhere in your post. The fact it has over 1200 lines makes me shiver straight away though!

[Stephen23]

Ida Lunde Hygum's "Answer" moved here:

Thank you for the quick reply! We are using globals because we define some parameters in one script which are used in a function. Can you use something else, so the function knows where to find the values? The reshape error comes from the transposing:

Extra.Q=(dataChassieu(1).flow)'

Will see if i can reduce my code to something that is easier to to figure out what is wrong with.

[Stephen23]

@Ida Lunde Hygum: Don't use globals. Just put the parameters into one structure and pass that as an input/output argument. The MATLAB documentation advises to: "use global variables sparingly, if at all."

source: https://www.mathworks.com/help/matlab/matlab_prog/share-data-between-workspaces.html

Your code will be less buggy and easier to debug without them.

[Rik]

The fact it has over 1200 lines makes me shiver straight away though!

@Adam, I have one script that is about 1750 lines (including comments and white space), and another that is even almost 4000 lines (including comments and white space). Long files aren't a problem, keeping your functions reasonably short is the goal. Although I must admit that last one is on the edge of not being manageable anymore. (the first has 23 functions inside 1 file, the later 33)

A huge file can also be an artifact of copying older versions of a function when updating. A function name of runDREAM_4 makes me suspect this might be the case.

(and yes, I know there's an important difference between script, function, and file, but I like to use script and file interchangeably)

[Adam]

Well, yes, now that scripts can have sub-functions, which they couldn't until recently, it is possible to have a > 1000 line script file that isn't a complete mess!

If it is just 1200+ lines of pure script though without sub-functions that really is unpleasant!