Extract data from sequentially named tables

So, the problem is the following: due to a sensor having a bad output file naming, my output data is now structured as tables with sequential names. I have around 20 subjects, with 12 files each, so I cannot call everything individually/manually. For each subject, there are 12 files: Myfile1.mat, Myfile2.mat, [...] Myfile12.mat. I load these automatically and save them to a struct :

for ii=1:length(subjies)
    i=subjies{ii}; 
    
    for jj=1:12
        ladeknecht{:,jj}=load (['E:\task\Knecht_' num2str(i) '\imported\Knecht_' num2str(i) '_' num2str(jj)])   
    end
end

The problem is the following. Each of the MyfileN.mat contains a table with the data, that has unfortunately been named sequentially by the sensor. Like this: Myfile1.mat contains Knecht0301, Myfile2.mat contains Knecht0302 [...]. So after loading the files, I am left with tables I cannot call, because they are named sequentially. I understand that sequentially naming things is frowned upon, again, the sensor output is structured like that and I need to extract the data anyways.

What I want to do, is to extract all contents from the table and save them in a struct / cell array or rename them so iI can automatically call them.

Thank you.

Answer 1

[Stephen23]

"So after loading the files, I am left with tables I cannot call, because they are named sequentially..."

Actually there is a really neat solution to this using STRUCT2CELL:

C = cell(..);
for k = ..
    C(k) = struct2cell(load(..));
end

Note that if there are multiple variables in any MAT file then the code will throw an error. Converting the structure to cell array is quite efficient (it basically just changes some pointers pointing to the actual data arrays), and completely avoids the issue of badly-sequentially-named variables.

[Stephen23]

Note that you can efficiently concatenate all of the tables together (just as Cris LaPierre suggests) after the loop:

T = vertcat(C{:})

[Lukas]

That worked and is the most elegant and fastest way. Also, I don't have to worry about calling this stuff wrong now. Perfect, thank you very much!

[Lukas]

Is there any drawback or bug risk to that solution?

[Stephen23]

Drawbacks: simpler, neater, more efficient code.

Bug Risk: hmmm, it really all depends on how well-specified your MAT files are. Because the imported variable names are completely discarded, any deviation from the expected variable names will not be noticed. For example, if a folder contains a special MAT file with a single "testparams" variable (e.g. with experiment parameters) then this will be imported too, without warning. Or for example, if the order of the variable names is incorrect, this will not be noticed (of course storing meta-data only in the variable name would be ... very poor data design).

Also if a MAT file does not contain exactly one variable then it will throw a slightly unhelpful error message (probably something about the indexing).

Nutshell: if you trust your own process to deliver only the files that you expect, then the code is quite acceptable.

[Lukas]

Perfect, that is exactly the case. All the pros, none of the cons you mentioned. Thanks

[Stephen23]

@Lukas: please remember to accept my answer if it helped you!

Answer 2

[Steven Lord]

Based on your code, ladeknecht is a cell array each cell of which contains a struct array. Let's create such a sample object on which to operate.

s = struct('x', 1, 'A', 2:3, 'Q', magic(3))

s = struct with fields:
    x: 1
    A: [2 3]
    Q: [3×3 double]

C = {s}

C = 1×1 cell array
    {1×1 struct}

Now, without hard-coding can I retrieve one of the fields of s?

s2 = C{1};
availableFields = fieldnames(s2)

availableFields = 3×1 cell array
    {'x'}
    {'A'}
    {'Q'}

A2 = s2.(availableFields{2})

A2 = 1×2

2 3

You could use this same technique on your data.

1. Extract the struct array from the cell.
2. Ask for the fieldnames of the struct array using fieldnames.
3. Use one of the field names you obtained in step 2 as a dynamic field name. You could also use getfield for the same purpose if you're unsure about dynamic field names. In the code above I always asked for the contents of the field A of the struct, but that's just for example purposes. There's nothing preventing me from asking for a random field.

data = getfield(s2, availableFields{3})

data = 3×3

8 1 6 3 5 7 4 9 2

whichField = randi(numel(availableFields));
fprintf("Retrieving field %s.\n", availableFields{whichField})

Retrieving field Q

data = s2.(availableFields{whichField})

data = 3×3

8 1 6 3 5 7 4 9 2

You may be tempted to create variable names Knecht0301, Knecht0302, etc. in your loop. Instead I'd store each of the data you extract from each of the cells into the same variable name (like I did with the last line of code above, where I used the name data.) That simplifies the remainder of your code; you can write it so it always operates on the variable data rather than trying to make them operate on a variable variable name.

Answer 3

[Cris LaPierre]

Converting your tables to stuct or cells to me seems like taking a step backward. Why not just concatenate all the tables together into a single table? Then you can access all the data from a single table. See the Access Data in Tables page for different ways to extract data from a table.

I assume the same variables exist in each table, so it should be just a matter of using square brackets to combine them. See this answer: https://www.mathworks.com/matlabcentral/answers/279758-join-two-tables-with-the-same-headers

If you don't know the name of your variables beforehand, you could try something like this.

data1 = load('Myfile1.mat')
data2 = load('Myfile2.mat')
varNames1 = fieldnames(data1)
varNames2 = fieldnames(data2)
T = data1.(varNames1{1});
T = [T;data2.(varNames2{1})]

You would probably ultimately want to put that in a loop, but this should get you started.

[Lukas]

Thank you, but this is not the way to go for me, as the 12 recordings per subject of the sensor must be seperate, as they need to be matched with 12 EEG recordings