create vector with number of hour per years

Hi everyone,

I should to create a vector with the number of hours per years. I already created the vector but its dimension is different by starting date vector and I don't know why considering that the vector has been created starting from the date vector. Can someone indicate me if is there an easier method to do this or can find the error? Thanks, I'm attacching the code below.

format long g
folderData = 'D:\Valerio\data\Cantabria_UPV_Analysis\data';
filePattern = fullfile(folderData, '*.xlsx');
xlsFiles = dir(filePattern);
nFiles = length(xlsFiles);

for ii = 1:nFiles
    filename = fullfile(xlsFiles(ii).folder, xlsFiles(ii).name);
    files{ii} = xlsread(filename);
end

ERA5 = files(1);
BUOY = files(2);

ERA5_data = unique(ERA5{:,1},'rows');
BUOY_data = unique(BUOY{:,1},'rows');

dt_BUOY = datetime([BUOY_data(:,1:3) BUOY_data(:,4) repmat([0 0],size(BUOY_data,1),1)]);
dt_ERA5 = datetime([ERA5_data(:,1:4) repmat([0 0],size(ERA5_data,1),1)]);
[~,ia,ie] = intersect(dt_BUOY,dt_ERA5);
tt_BUOY_ERA5 = timetable(dt_BUOY(ia),BUOY_data(ia,5:end),ERA5_data(ie,5:end));
tt_BUOY_ERA5.Properties.VariableNames = {'BUOY','ERA5'};

BUOY = tt_BUOY_ERA5.BUOY;
ERA5 = tt_BUOY_ERA5.ERA5;
time = tt_BUOY_ERA5.Time;

x1 = ERA5(:,1); %Hs from ERA5
Tp_ERA5 = ERA5(:,2); %Peak period from ERA5
dir_ERA5 = ERA5(:,3); %Dir from ERA5

y = BUOY(:,1); %Hs from BUOY
Tp_BUOY = BUOY(:,3); %Tp from BUOY
dir_BUOY = BUOY(:,7); %Dir from BUOY
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2005%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_05 = timerange('01-Jan-2005 00:00:00','01-Jan-2006 00:00:00');
tt_05 = tt_BUOY_ERA5(TR_05,:);
time_05 = tt_05.Time;
start_1 = datenum('01-Jan-2005 00:00:00');
start_2 = datenum(time_05(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_05(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours05 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2006%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_06 = timerange('01-Jan-2006 00:00:00','01-Jan-2007 00:00:00');
tt_06 = tt_BUOY_ERA5(TR_06,:);
time_06 = tt_06.Time;
start_1 = datenum('01-Jan-2006 00:00:00');
start_2 = datenum(time_06(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_06(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours06 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2007%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_07 = timerange('01-Jan-2007 00:00:00','01-Jan-2008 00:00:00');
tt_07 = tt_BUOY_ERA5(TR_07,:);
time_07 = tt_07.Time;
start_1 = datenum('01-Jan-2007 00:00:00');
start_2 = datenum(time_07(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_07(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours07 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2008%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_08 = timerange('01-Jan-2008 00:00:00','01-Jan-2009 00:00:00');
tt_08 = tt_BUOY_ERA5(TR_08,:);
time_08 = tt_08.Time;
start_1 = datenum('01-Jan-2008 00:00:00');
start_2 = datenum(time_08(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_08(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours08 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2009%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_09 = timerange('01-Jan-2009 00:00:00','01-Jan-2010 00:00:00');
tt_09 = tt_BUOY_ERA5(TR_09,:);
time_09 = tt_09.Time;
start_1 = datenum('01-Jan-2009 00:00:00');
start_2 = datenum(time_09(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_09(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours09 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2010%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_10 = timerange('01-Jan-2010 00:00:00','01-Jan-2011 00:00:00');
tt_10 = tt_BUOY_ERA5(TR_10,:);
time_10 = tt_10.Time;
start_1 = datenum('01-Jan-2010 00:00:00');
start_2 = datenum(time_10(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_10(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours10 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2011%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_11 = timerange('01-Jan-2011 00:00:00','01-Jan-2012 00:00:00');
tt_11 = tt_BUOY_ERA5(TR_11,:);
time_11 = tt_11.Time;
start_1 = datenum('01-Jan-2011 00:00:00');
start_2 = datenum(time_11(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_11(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours11 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2012%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_12 = timerange('01-Jan-2012 00:00:00','01-Jan-2013 00:00:00');
tt_12 = tt_BUOY_ERA5(TR_12,:);
time_12 = tt_12.Time;
start_1 = datenum('01-Jan-2012 00:00:00');
start_2 = datenum(time_12(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_12(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours12 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2013%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_13 = timerange('01-Jan-2013 00:00:00','01-Jan-2014 00:00:00');
tt_13 = tt_BUOY_ERA5(TR_13,:);
time_13 = tt_13.Time;
start_1 = datenum('01-Jan-2013 00:00:00');
start_2 = datenum(time_13(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_13(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours13 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2014%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_14 = timerange('01-Jan-2014 00:00:00','01-Jan-2015 00:00:00');
tt_14 = tt_BUOY_ERA5(TR_14,:);
time_14 = tt_14.Time;
start_1 = datenum('01-Jan-2014 00:00:00');
start_2 = datenum(time_14(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_14(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours14 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2015%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_15 = timerange('01-Jan-2015 00:00:00','01-Jan-2016 00:00:00');
tt_15 = tt_BUOY_ERA5(TR_15,:);
time_15 = tt_15.Time;
start_1 = datenum('01-Jan-2015 00:00:00');
start_2 = datenum(time_15(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_15(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours15 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2016%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_16 = timerange('01-Jan-2016 00:00:00','01-Jan-2017 00:00:00');
tt_16 = tt_BUOY_ERA5(TR_16,:);
time_16 = tt_16.Time;
start_1 = datenum('01-Jan-2016 00:00:00');
start_2 = datenum(time_16(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_16(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours16 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2017%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_17 = timerange('01-Jan-2017 00:00:00','01-Jan-2018 00:00:00');
tt_17 = tt_BUOY_ERA5(TR_17,:);
time_17 = tt_17.Time;
start_1 = datenum('01-Jan-2017 00:00:00');
start_2 = datenum(time_17(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_17(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours17 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2018%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_18 = timerange('01-Jan-2018 00:00:00','01-Jan-2019 00:00:00');
tt_18 = tt_BUOY_ERA5(TR_18,:);
time_18 = tt_18.Time;
start_1 = datenum('01-Jan-2018 00:00:00');
start_2 = datenum(time_18(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_18(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours18 = (start_hours:end_hours).';
%%%%%%%%%%%%%%%%%%%%HOURS NUMBER IN 2019%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
TR_19 = timerange('01-Jan-2019 00:00:00','31-Dec-2019 23:00:00');
tt_19 = tt_BUOY_ERA5(TR_19,:);
time_19 = tt_19.Time;
start_1 = datenum('01-Jan-2019 00:00:00');
start_2 = datenum(time_19(1,:));
start = start_2 - start_1;
start_hours = floor(start * 24);
end_one = datenum(time_19(end,:));
Dt = end_one - start_1;
end_hours = floor(Dt * 24);
hours19 = (start_hours:end_hours).';
hours_YY = 8766.00;
HH = [hours05; hours06; hours07; hours08; hours09; hours10; hours11; hours12; hours13; hours14; hours15; hours16; hours17; hours18; hours19];
time_dir = HH/hours_YY;
x2 = time_dir * 360;

I can't attach the data because are too much heavy.

[dpb]

Way too much code to try to wade through, sorry...

Either break it down to the minimum to reproduce the problem with a small dataset or describe more succinctly what you're trying to create and what the problem you ran into is.

First off, however, don't use the obsolete datenum, use datetime and duration instead.

You also could find timetable very helpful or maybe timeseries altho I've not found a real use for the latter so far that the former doesn't do as easily.

[Steven Lord]

I haven't read through the code, but my suspicion is that either 1) you're missing data for one or more of the hours in your original data set that you're trying to split into year-long pieces or 2) you've forgotten that certain years (those that are multiples of 4 but not also multiples of 100 unless they're multiples of 400, aka leap years) have more hours than others.

[darova]

I didn't understand your question but the only thing i know is that your code can be much shorter

Use for loop. Example:

% TR_05 = timerange('01-Jan-2005 00:00:00','01-Jan-2006 00:00:00');
% tt_05 = tt_BUOY_ERA5(TR_05,:);
for i = 1:14
    s1 = sprintf('01-Jan-%d 00:00:00',2004+i);
    s2 = sprintf('01-Jan-%d 00:00:00',2005+i);
    TR = timerange(s1,s2);
    tt = tt_BUOY_ERA5(TR,:);
end

[Valerio Gianforte]

To be clearer Ihave to do a vector with number of hours corrisponding at the dates. The starting point is 01-Jan-2005 but the date vector start from 15-Sep-2005 16:00:00, so the new vector should have the same dimensions of the the date vector and it has to contain number of hour corrisponding of each date of year. For example 01-Jan-2005 00:00:00 in the new vector must be 1h while 31-Dec-2005 23:00:00 must be 8759 and every year have to start from 1 if the year start from 01-Jan or by the hours number corrisponding at the date. I hope that I have been clearer than the question. Thanks

Answer 1

[dpb]

Oh. I see at the beginning you did create a timetable -- good! Just use it--

tt_BUOY_ERA5.Properties.DimensionNames={'Time','Variables'};        % rename time column for convenience
tt_BUOY_ERA5.ElapsedTime=tt_BUOY_ERA5.Time-tt_BUOY_ERA5.Time(1);    % create the elapsed time as duration
tt_BUOY_ERA5.Hours=hours(tt_BUOY_ERA5.ElapsedTime);                 % elapsed time as double in hours

will leave you with the times associated with the actual data accounting for any missing...

ADDENDUM:

As noted, eyes glazed over with all the code so thought were trying to build a vector from beginning...to do for each year is also pretty simple--Steven showed the way altho went on to demonstrate another feature or two...

 tt_BUOY_ERA5.AnnualHrs=hours(tt_BUOY_ERA5.Time-dateshift(tt_BUOY_ERA5.Time,'start','year'));

produces variable that is hours from beginning of the year irrespective of date of first observation within the year.

[dpb]

BTW, just the number of hours/year without any possibly missing dates in the data is

 24*(365+isleapyr(2005:2019))

where isleapyr is my utility function:

function is=isleapyr(yr)
%  returns T for input year being a leapyear

  is=(eomday(yr,2)==29);
end

[Valerio Gianforte]

Thanks but that's no that I have to do. The first code that you posted is a good solution for my problem but my new vector for each year has to start from 1 if the starting date is 01-Jan 00:00:00 otherwise it has to start from corrisponding number hours at that date, therefore it has to finish with 8759 if the finishing date is 31-Dec 23:00:00 otherwise with corrisponding number hours at finishing date. Thanks

[dpb]

See amended Answer...so much code I thought you were appending vectors from beginning...

As far as zero- or one-based counting, that's simply an adjustment of +1 or not...

Answer 2

[Steven Lord]

If you need to extract and operate on each year's worth of data separately the timerange and retime functions will likely be of use to you. Let's make some sample data.

rng default
t = datetime('today');
time = t + days(randi([0 1000], 10, 1));
data = (1:10).';
tt = timetable(time, data)

If I want to extract just data from this year, first I find the start of this year using dateshift.

startThisYear = dateshift(t, 'start', 'year');

Next I make a timerange that covers just this calendar year.

thisYear = timerange(startThisYear, startThisYear+calyears(1))

Finally I use the timerange to index into the timetable tt.

tt(thisYear, :)

As I type this tt has two rows whose times are in 2020, rows 3 and 6, and those are the ones returned by that indexing operation.

If I wanted to aggregate the data by years, I could do that with retime. For example, taking the yearly mean:

retime(tt, 'yearly', 'mean')