Filtering Data

Hello,

I am trying to compute an average and variance of a large data set (~330000 rows x 3 different columns), but I'd like to take the average and var. of every 240 entries. I think you'd call this a moving average/variance, but I'm not sure.

Can you please provide hints as to accomplish this? I'm thinking either a for-loop or using the 'filter' function, but am not sure how to approach this problem...

Thanks in advance,

Nic

[ndb]

Also,

I already have a column that records the minute each measurement is made, there being 4 measurements taken each second (4 * 60 = 240 measurements a minute). So I'm thinking that I might be able to construct a loop that looks to see if the minute column has changed (say, from minute 0 to minute 1) and if so, average/take variance of the values stored in some temporary variable...

Am I on the right track?

[Andrew Newell]

By "entries" do you mean rows or total number of variables?

Answer 1

[Jarrod Rivituso]

Here's a filter function example for ya...

Doc reference: http://www.mathworks.com/help/techdoc/ref/filter.html

Example code for simple moving average:

data = rand(4000,3);
window = ones(240,1)/240;
filtered_data = filter(window,1,data);

Hope this helps!

[Jarrod Rivituso]

After reading your additional comment, here's a fancy bit of code that you might find useful:

%create random data

data = rand(100,4);

%emulate 4th column a bit

data(1:10,4) = 1;

data(11:20,4) = 2;

%extract all rows where 4th column is equal to 1

extractedData = data(data(:,4) == 1,:)

%find variance of extracted data

var(extractedData)