How to load multiple txt files with many rows and 4 columns and get mean of 3rd and 4th columns

Hi I have 400 txt files with many rows, 4 columns and 9 header lines and columns are separated by spaces. How can load them all in matlab and get mean of third and fourth columns.

Answer 1

[Walter Roberson]

projectdir = '.';   %absolute path would be even better
dinfo = dir( fullfile(projectdir, '*.txt'));
filenames = fullfile({dinfo.folder}, {dinfo.name});
numfiles = length(filenames);

data = cell(numfiles, 1);
for K = 1 : numfiles
    thisfile = filenames{K};
    thisdata = readmatrix(thisfile, 'HeaderLines', 9);
    data{K} = thisdata;
end

After that it depends whether the mean over the columns is intended to be per-file (column3+column4)/2, or per file [mean(column3), mean(column4)], or over all of the files (mean of each element between all of the files.) If you intend mean over all of the files, then you would have to require that the files are all the same size, or else you would have to define what you want to do for the case of files that have fewer rows.

[SK]

Thankyou for the answer Walter. As for the mean I want the mean of 3rd column and mean of 4th column over all files and the files have the same size

[Walter Roberson]

projectdir = '.';   %absolute path would be even better
dinfo = dir( fullfile(projectdir, '*.txt'));
filenames = fullfile({dinfo.folder}, {dinfo.name});
numfiles = length(filenames);

output = zeros(numfiles, 2);
for K = 1 : numfiles
    thisfile = filenames{K};
    thisdata = readmatrix(thisfile, 'HeaderLines', 9);
    mean34 = mean(thisdata(:,3:4), 1);
    output(K, :) = mean34;
end

[SK]

Thankyou so much it works perfectely! Can you help me with another problem im facing that i want to extract the data from the first row of third column of each file and store it to plot it against time but i cant figure out how to do it. Sorry for asking many questions I'm a newbie to matlab

[Walter Roberson]

projectdir = '.';   %absolute path would be even better
dinfo = dir( fullfile(projectdir, '*.txt'));
filenames = fullfile({dinfo.folder}, {dinfo.name});
numfiles = length(filenames);

output = zeros(numfiles, 2);
row31 = zeros(numfiles, 2);
times = zeros(numfiles,1);
for K = 1 : numfiles
    thisfile = filenames{K};
    thisdata = readmatrix(thisfile, 'HeaderLines', 9);
    mean34 = mean(thisdata(:,3:4), 1);
    output(K, :) = mean34;
    row31(K) = thisdata(1,3);
    times(K) = SOMETHING;
end

plot(times, row31)

unfortunately I do not have any information about where the time information is located. I would not be surprised if the time is stored as part of the file name but if so then we would need more information about that, and we would have to be concerned about whether the order returned by dir() is the same as increasing order of time.

[SK]

Thankyou again! The time is in the header lines but its increasing in increments of 0.025s over 399 files so it goes from 0 - 9.95s so I can just do a linspace

[Walter Roberson]

But can you be certain the the file order returned by dir() is the same as increasing time?

In practice the file order is pretty much ASCII sort, first character then second character, and so on. So file1.txt file2.txt ... file9.txt file10.txt ... The file10.txt would probably show up between file1.txt and file2.txt because the '1' of '10' sorts before '2'. If you have file names with leading zeros you typically do not need to take extra care (well, unless the numbers are dates, then you can have problems with date/day/year formats)

Loading a time out of a header can be done, sure, but the difficulty of doing that depends on what the headers look like.

[SK]

Yes I checked the the file order is the same as the increasing time.

[Walter Roberson]

times(K) = 0.025 * (K-1);