Import .csv Data and assemble structure array

Hey everyone!

I have a folder with a lots of .csv data, always with the same pattern (just changing ID-number):

logfile_11111_accelerationX.csv, logfile_11111_accelerationY.csv,...,logfile_11111_speed.csv

logfile_22222_accelerationX.csv, logfile_22222_accelerationY.csv,...,logfile_22222_speed.csv

I also have a meta file where all those ID-numbers (11111,22222,...) are specified.

What I want to do is to write a function which assembles all the .csv-Data with the same ID-number to one .mat struct, e.g. logData11111, where all data is included, e.g. logData11111.accelerationX, logData11111.speed, and so on.

Does someone have an idea how to do this?

Thanks in advance!

[Stephen23]

What exactly is a ".mat struct"? I know what a .mat file is, and I know what a structure is, but I have never heard of a ".mat struct". Do you want the data saved in a .mat file, or in a structure in MATLAB memory?

Answer 1

[Adam Danz]

First list all files in your directory. This will get you started:

dirContent = dir('C:\Users\blah\blah\blah');
contentList = {dirContent.name};

Then loop through each file ID number and use regexp or strfind to identify matching files.

fileIDs = {'1111', '2222', '3333'};  %from your meta file 
for i = 1:length(fileIDs)
    matches = regexp(contentList, sprintf('%s*.csv', fileIDs{i}), 'match');
    selectedFiles = matches{~cellfun(@isempty, matches)};
    % Now you can import data from selectedFiles{:} here

end

My code assumes your fileIDs will be strings; if that's not the case you'll need to make small changes. You may also need to adapt the regular expression to your filenames. This has not been tested and represents a basic method you'll likely need to adapt to your needs.

Answer 2

[Stephen23]

Two tested and working solutions for loading the CSV data and saving it into mat files (which is one interpretation of your request to "...assembles all the .csv-Data with the same ID-number to one .mat struct")

Method one: load each group of data into a structure, then save as a .mat file.

S = dir('logfile*.csv');
N = sort({S.name});
T = regexp(N,'^logfile_(\d+)_(\w+)\.csv$','tokens','once');
F = cellfun(@(c)c{2},T,'uni',0);
U = cellfun(@(c)c{1},T,'uni',0);
[U,~,X] = unique(U);
for ii = 1:max(X)
    S = struct();
    for jj = find(ii==X);
        S.(F{jj}) = csvread(N{jj});
    end
    Z = sprintf('logData%s.mat',U{ii});
    save(Z,'-struct','S')
end

You could easily adapt this to import all of the data into one structure array (perhaps this is what you mean by "assemble structure array"?), by replacing the two loops with this:

Z = struct('ID',U)
for ii = 1:max(X)
    for jj = find(ii==X);
        Z(ii).(F{jj}) = csvread(N{jj});
    end
end

Method two: load just one file into memory at a time, append to the .mat files:

S = dir('logfile*.csv');
N = sort({S.name});
T = regexp(N,'^logfile_(\d+)_(\w+)\.csv$','tokens','once');
F = cellfun(@(c)c{2},T,'uni',0);
U = cellfun(@(c)c{1},T,'uni',0);
C = {'-append'};
for k = 1:numel(N)
    S = struct();
    S.(F{k}) = csvread(N{k});
    Z = sprintf('logData%s.mat',U{k});  
    save(Z,'-struct','S',C{exist(Z,'file')==2})
end

The test files are attached.