How to calculate 8 days mean from ten years data?
3 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
I have a code which calculates the 8days mean from single year. It computes 45 eight days from 45X8 = 360 days and last 8days mean from remaining 5 or 6 days of the year. I wish to use this code over ten years data with the same method as mentioned in previous line. It should not jump to next year to complete last 8 days. It should start from the 1st January and finish at 31 December of each year and then keep on doing similar way so till the end of all the data. I request you all to kindly suggest me some checks in this code to run this code over a length of continuous ten years data without overlapping between years. The code is as follows
data_matrix=readmatrix('c:/data/test.csv');
start = 1:8:height(data_matrix);
stop = unique([start(2:end)-1, height(data_matrix)]);
nbins = numel(stop);
dataAverages = nan(nbins, width(data_matrix));
for i = 1:nbins
rows = start(i) : stop(i);
dataAverages(i,:) = mean(data_matrix(rows, :), 1, 'omitnan');
end
disp(dataAverages)
size(dataAverages)
% Write output as matrix in csv format
filename = 'c:/data/test_out.csv'
writematrix(dataAverages,filename);
Devendra
12 commentaires
Mathieu NOE
le 21 Juil 2023
I am just seing that I actually didn't post my answer in the answer section
so I am gonna move it , so you could accept it if you want !
Réponse acceptée
Mathieu NOE
le 21 Juil 2023
Déplacé(e) : Mathieu NOE
le 21 Juil 2023
ok so we have to do the processing for each year and pay attention to the remaining (variable length) days
NB that as we now have access to each year results , you can decide wheteher you want to store them in a separate file
for the time being it's doing a vertical concatenation of all results
to load the data I used this fex submission
which gave me a fast loading , but of course there are plenty of alternatives if you have another preference
i also tested the case where the recording would stop randomly before a 31 december
for the provided file Test.csv, I have 138 data points as expected
%% read data file
[lines, ~] = readlog('Test2.csv'); % fex : https://fr.mathworks.com/matlabcentral/fileexchange/96364-readlog?s_tid=srchtitle
data_matrix = split(lines,',');
[m,n] = size(data_matrix);
ind_start_year = find(startsWith(data_matrix(:,1),'1-Jan-')); % find row corresponding to first day of the year
ind_stop_year = find(startsWith(data_matrix(:,1),'31-Dec-')); % find row corresponding to last day of the year
toc
%% main loop
windowSize = 8; % define buffer length for averaging
cols = 3:n; % define cols to be processed
dataAverages_total = [];
% if the data file stops before a 31 Dec , we must add this :
if numel(ind_stop_year) < numel(ind_start_year)
ind_stop_year = [ind_stop_year; m];
end
for ck = 1:numel(ind_start_year)% loop over year number
ind_start_thisyear = ind_start_year(ck);
ind_stop_thisyear = ind_stop_year(ck);
start = ind_start_thisyear:windowSize:ind_stop_thisyear;
stop = [start(2:end)-1, ind_stop_thisyear];
dur = stop-start+1;
nbins = numel(start);
dataAverages = nan(nbins, numel(cols));
for ci = 1:nbins
rows = start(ci) : stop(ci);
tmp = data_matrix(rows, :);
dataAverages(ci,:) = mean(str2double(tmp(:,3:n)), 1, 'omitnan'); % results of one year
end
sum(dur)
dataAverages_total = [dataAverages_total ; dataAverages]; % concatenate every year's result (dataAverages)
end
0 commentaires
Plus de réponses (1)
Mathieu NOE
le 21 Juil 2023
Déplacé(e) : Mathieu NOE
le 21 Juil 2023
you can use the existing standard readlines function instead of readlog as shown above
lines = readlines('Test.csv','EmptyLineRule','skip');
but I had to add 'EmptyLineRule','skip' to avoid empty (last) line being loaded
Voir également
Catégories
En savoir plus sur Symbolic Variables, Expressions, Functions, and Preferences dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!