How to create a for loop to calc average of datas in a STRUCTURE?

Hi, I have created a structure that splits my data with regards to specific loads. I was able to combine the repeats so all my data is in one structure.

I now want to calc averages so i can plot the data.

I have used

CoF1= [data_all{1, 2}(:, 5), data_all{1, 4}(:, 5), data_all{1, 6}(:, 5)]; %to have data together

CoF_av1 = mean(CoF1')'; %to get data across the rows

CoF_avall1 = mean(CoF1,'all'); %to get one average value fro all data

However as you can see the data splits up 10 times for this set and I would like to to create a for loop that will do it easier for me.

Therefore, my question is how would I create a loop to just average the 3 sets of data for me from within a structure.

So from the image I want to find out the average for each colum across each embedded structure-2, 4 and 6 column from 'data_all'.

Such that I am left with one data file that is the average of these 3, where each colum is a result of the average of my repeats. and then have an additional row that averages the column.

Answer 1

[Guillaume]

First, note that in matlab, a structure is particular data type. Here you're using structure with its non-matlab meaning. It can be a bit confusing. Your data is actually stored in a cell array.

Also note that mean(x')' is not a efficient way to calculate the mean across the rows. mean(x, 2) is a lot more efficient.

My initial reaction to your question is that the way you're storing your data is going to lead to complicated code and maybe we should be focusing on changing that rather than your actual question. In particular, I don't understand why you're repeating column 1 as column 3 and 5. You mention that you've grouped/split your data according to a criteria. Often it's better not to group/split the data and use a flatter storage format. If you explain in more details what you started with, we can look into that.

As it is, to do "Essentially i want to create another structure that is split in the same way. such that i have a column that is same as 1,3 and 5. I then want to create structure split this way but each little box includes 100*8 columns of Averages of the 3 datas i have", this is how I'd go about it:

%It is assumed that columns 1,3 and 5 are identical
assert(isequal(data_all(:, 1), data_all(:, 3)) && isequal(data_all(:, 3), data_all(:, 5)), 'Column 1,3 and 5 are not identical')
result = [data_all(:, 1), ...
          cellfun(@(col2, col4, col6) mean(cat(3, col2, col4, col6), 3), ...
                  data_all(:, 2), data_all(:, 4), data_all(:, 6), ...
                  'UniformOutput', false)]

[Sufia Fatima]

Your most likly right about the way im storing the data. However this is a much smaller example when i have raw data i have millions of points. which i need to split in respect to one specfific thing ie load or speed. This is why i have it this way. being fairly new at MATLAB im not sure how else i would do it.

Thank you so much for your help this worked vey well.

[Sufia Fatima]

@ Guillaume

Hi, againa thanks for help earlier. I have used your code to do the same thing and calc st.dev this time, so I basically used what you did but used 'std' function but got the follwoing error;

Error in std (line 59)

y = sqrt(var(varargin{:}));

Error in splitload10_100>@(col2,col4,col6)std(cat(3,col2,col4,col6),3)

Error in splitload10_100 (line 63)

cellfun(@(col2, col4, col6) std (cat(3, col2, col4, col6), 3), ...

Tbh im not sure what the error means do i need to find variance first?

If you could help that would be great.

------------------------------------------------------------------------

[Guillaume]

What mean(cat(3, col2, col4, col6), 3) does is:

• concatenate the 3 matrices in the 3rd dimension (with cat(3, ...))
• calculate the mean across that 3rd dimension (with mean(..., 3))

You can use the same principle for std, however the dimension input of std is the 3rd input. The second input specifies which normalisation you want to use, therefore:

std(cat(3, col2, col4, col6), 0, 3)   %0 for default = N-1 normalisation

Answer 2

[Bob Thompson]

I apologize, I don't quite understand what you're looking for as a final product. Would you mind restating it, and maybe having a small example?

To loop through each of the rows of data_all is fairly simple, just index the row within your calling of CoF1 and other equations.

for i = 1:size(data_all,1);
    CoF1 = [data_all{i, 2}(:, 5), data_all{i, 4}(:, 5), data_all{i, 6}(:, 5)];
    CoF_av1(:,i) = mean(CoF1')';
    CoF_avall1(i) = mean(CoF1,'all');
end

Alternatively, if you're looking for a single set of averages for all of the data in each column, you can combine the data into a single array and then average. You may have to do a loop, but I would try vertcat first.

datcomb = vertcat([data_all{:, 2}(:, 5), data_all{:, 4}(:, 5), data_all{:, 6}(:, 5)]);

% Or the loop method
datcomb = [];
for i = 1:size(data_all,1);
    datcomb = [datcomb; [data_all{1, 2}(:, 5), data_all{1, 4}(:, 5), data_all{1, 6}(:, 5)]]
end

After combining then you can run your averaging the same way you have them set up.

[Sufia Fatima]

Hi

Where you have done it for just column 5 i wanna create a loop where it does it for all 1-8 columns in each field.

Essentially i want to create another structure that is split in the same way. such that i have a column that is same as 1,3 and 5. I then want to create structure split this way but each little box includes 100*8 columns of Averages of the 3 datas i have. So i wanna average along these rows and along the indiviidual colums.

[Bob Thompson]

To address the first part. You can loop through the individual columns of the data sets with a second nested loop.

for i = 1:size(data_all,1);
    for j = 1:size(data_all{1,2},2)
    CoF1 = [data_all{i, 2}(:,j), data_all{i, 4}(:, j), data_all{i, 6}(:, j)];
    CoF_av1(:,j) = mean(CoF1')'; % Not it no longer retains different rows of data_all. Adjust as desired
    CoF_avall1(i,j) = mean(CoF1,'all');
end

The way I understand what you're saying, you want a new cell array with columns 1, 3, and 5 duplicating the original 1, 3, and 5 columns, and columns 2, 4, and 6 being the CoF_av1 results. Is this correct? If so, here is how to do it.

for i = 1:size(data_all,1);
    for j = 1:size(data_all{1,2},2)
    CoF1 = [data_all{i, 2}(:,j), data_all{i, 4}(:, j), data_all{i, 6}(:, j)];
    CoF_av1(:,j) = mean(CoF1')'; % Not it no longer retains different rows of data_all. Adjust as desired
    CoF_avall1(i,j) = mean(CoF1,'all');
end

[Guillaume]

I don't particularly understand why you iterate over the columns of the individual matrices. Seems like a complete waste of time. Just concatenate the 3 matrices in the 3rd dimension and take the mean across that.

As I said in my answer, mean(CoF')' is very inefficient. transpose is a very expensive operation. Since the position of every element changes in memory, they must be copied one at a time. mean(Cof, 2) is both clearer and a lot faster. On my computer, it's an order of magnitude faster.

[Sufia Fatima]

@Bob Nbob thank you also for your help. I had someone give me a similar suggestoion unfortunatly this did not work well when calculating the mean. Some error regarding indexing came coming up. I basically used that with just adding and dividing by 3, which worked but isnt efficient.

Thank you.