Appending data when extracting nested fields as individual variables

Hi Matlab community,

I have a variable "hands" that contains several fields, and I am trying to make hand_type, one of the fields, as an individual variable. The size of "hands" varies between 1x1 struct and 1x2 struct, hence I used the if-elseif loop. When "hands" is 1x2 struct, I want the second group of data to be stored in a parallel cell next to the first group of data, hence the {end+1}, as I don't want them to be in the same cell and create an additional layer of nesting.

However, in the resulting "hand_type", whenever "hands" is 1x2 struct, "hand_type" only has 0x0 double. In comparison, when "hands" is 1x1 struct, "hand_type" has a value of a number (0, 1, etc).

Could someone help me to identify what I am doing wrong, and how I can fix the problem? The full script is attached.

Thank you in advance!

has_hands = ~cellfun(@isempty, {frames.hands});
filtered_frames = frames(has_hands);
id = {filtered_frames.id};
time = {filtered_frames.timestamp};
hands = {filtered_frames.hands};
 
hand_type = cell(size(hands));
 
for i = 1:numel(hands)
    if isequal(size(hands{i}),[1,1])
        hand_type{i} = hands{i}.type;
 
    elseif isequal(size(hands{i}),[1,2])
            hand_type{end+1} = hands{i}(1).type;
            hand_type{end+1} = hands{i}(2).type;
end

[Voss]

Please save frames in a .mat file

save('frames.mat','frames')

and upload the .mat file using the paperclip button.

[Julia]

@Voss, I have uploaded the frames.mat file.

[Voss]

@Julia Thanks for that!

However, it looks like the variable in the .mat file is not similar to what you describe in the question.

S = load('frames.mat')

S = struct with fields:
    first100: [1x100 struct]

frames = S.first100

frames = 1x100 struct array with fields:
    frames

temp = [frames.frames]

temp = 1x100

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

any(temp)

ans = logical
   0

It is a struct array with one field (called "frames") containing all scalar zeros, instead of a struct array with multiple fields ("hands", "id", "timestamp").

[Julia]

@Voss I'm so sorry about that! I made a mistake when extracting part "frames" (the original variable is too large to upload). The correct version of frames.mat is now uploaded, which should represent what I described in the question.

Answer 1

[Stephen23]

The simple and efficient MATLAB approach using two comma-separated lists (no loops are required!):

first100 = load('frames.mat').first100

first100 = 1x100 struct array with fields:
    id
    timestamp
    hands
    version

tmp = [first100.hands]; % 1x140 structure array
hand_type = [tmp.type]  % 1x140 double array

hand_type = 1x140

0 0 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

https://www.mathworks.com/matlabcentral/discussions/tips/847976-tutorial-comma-separated-lists-and-how-to-use-them

[Stephen23]

" If I also need to double the values in "id" and "time" when "hands" has size [1,2], so that the data in "hands" can still match the id and timestamp, is it possible to do?"

Which ID field do you want: first100.id or first100.hands.id ? Unclear information delays getting answers.

In any case, here are both of them:

first100 = load('frames.mat').first100

first100 = 1x100 struct array with fields:
    id
    timestamp
    hands
    version

V = arrayfun(@(s)numel(s.hands),first100);
first_id = repelem([first100.id],V)

first_id = 1x140

108951 108952 108953 108954 108954 108955 108955 108956 108956 108957 108957 108958 108958 108959 108959 108960 108960 108961 108961 108962 108962 108963 108963 108964 108964 108965 108965 108966 108966 108967

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

format long G
timestamp = repelem([first100.timestamp],V)

timestamp = 1x140

1.0e+00 * 78820983095 78820992158 78821001174 78821010146 78821010146 78821019299 78821019299 78821028390 78821028390 78821037336 78821037336 78821046485 78821046485 78821055376 78821055376 78821064473 78821064473 78821073525 78821073525 78821082524 78821082524 78821091650 78821091650 78821100667 78821100667 78821109562 78821109562 78821118712 78821118712 78821127917

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

tmp = [first100.hands]; % 1x140 structure array
hands_id = [tmp.id]

hands_id = 1x140

54 54 54 54 55 54 55 54 55 54 55 54 55 54 55 54 55 54 55 54 55 54 55 54 55 54 55 54 55 54

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Note that a more useful way to store that timestamp is as a DATETIME array:

https://www.mathworks.com/help/matlab/ref/datetime.html

[Julia]

I can't thank you enough for your help! This is magical! I absolutely learnt a lot from your answers and guidance. Also sorry for the lack of clarity in my question; I wasn't aware of the repeat in variable names.

If I may, can I just ask one more question?

first100 = load('frames.mat').first100

Error using load
Unable to find file or directory 'frames.mat'.

tmp = [first100.hands]; 
tmppalm = [tmp.palm];

When I try to get data from "palm" under "hands", there are several numbers in one sub-variable. For example, "position" has 3 numbers in each cell. Therefore, I can't simply use palm_position = [tmppalm.position]; to extract data.

Is it possible for me to store all the 1st values of "position" to positionX, all the 2nd values to positionY, and all the 3rd values to positionZ?

[Stephen23]

"Is it possible for me to store all the 1st values of "position" to positionX, all the 2nd values to positionY, and all the 3rd values to positionZ?"

first100 = load('frames.mat').first100

first100 = 1x100 struct array with fields:
    id
    timestamp
    hands
    version

tmp = [first100.hands];
plm = [tmp.palm]

plm = 1x140 struct array with fields:
    position
    stabilized_position
    velocity
    normal
    width
    direction
    orientation

mat = vertcat(plm.position);
posX = mat(:,1)

posX = 140x1

-117.2521 -118.6039 -121.1529 -122.1259 -87.0131 -123.5399 -88.1751 -123.9327 -89.1215 -124.1052

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

posY = mat(:,2)

posY = 140x1

122.5035 121.4406 123.3779 125.7938 369.7952 127.7411 374.7938 129.2533 378.2943 130.0268

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

posZ = mat(:,3)

posZ = 140x1

-80.5295 -79.4773 -77.5457 -76.9199 -57.4891 -75.7636 -59.2263 -75.8140 -60.5207 -76.0107

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

plot3(posX,posY,posZ)

You could do this yourself if you:

• read the links I gave you, and
• keep a clear understanding of what sizes and classes of data you have.

[Julia]

Thank you so much for bearing with me! I have limited experience in Matlab or coding in general, and definately need to enhance my understanding to the properties of my data.

[Stephen23]

@Julia: none of us are born knowing what a cell array is! With container arrays (e.g. cell, struct, string) it is very important to understand the clear distinction between the container array itself and its contents (which are also arrays).

Although you wrote in your question that you wanted "cells", your numeric data should be stored in a numeric array (which is what my answer&comments give you). Storing lots of numeric scalars in a cell array makes processing the numeric data complex and inefficient. Numeric arrays are for storing numeric data and make it easy to process, e.g. by performing lots of mathematical operations on the entire numeric array at once:

https://www.mathworks.com/help/matlab/numeric-types.html

You might find this useful: https://www.mathworks.com/matlabcentral/discussions/tips/847916-experts-of-matlab-how-did-you-learn-any-advice-for-beginner-intermediate-users

[Julia]

Edited: Sorry for bothering you! I think I figured it out by using bones_only = arrayfun(@(x) x.bones, thumb, 'UniformOutput', false);

Hi@Stephen23, can I ask for your help one more time?

I am trying to extract data from "digits" in "hands". For my new variable "thumb_1", I would need all data of "prev_joint" in row 1 for each "bones" structure. However, I am having troubles accessing that.

Here are the steps I have taken so far:

digits = [tmp.digits];
thumb = digits([digits.finger_id] == 0);

Then, when I tried to extract "bones", thumb_data = thumb.bones; only gave me a 1x4 struct instead the whole array of data. I tried vertcat and indexing, which had the same problem.

I'm really sorry if this is another stupid question.

[Stephen23]

"For my new variable "thumb_1", I would need all data of "prev_joint" in row 1 for each "bones" structure. However, I am having troubles accessing that."

In total there are 2700 BONES structures: perhaps you only want to access a subset of them.

"Row 1" is confusing me... I do not see any arrays which have more than one row.

first100 = load('frames.mat').first100

first100 = 1x100 struct array with fields:
    id
    timestamp
    hands
    version

tmp = [first100.hands]

tmp = 1x140 struct array with fields:
    id
    type
    confidence
    visible_time
    pinch_distance
    grab_angle
    pinch_strength
    grab_strength
    palm
    digits
    arm

digits = [tmp.digits]

digits = 1x700 struct array with fields:
    finger_id
    is_extended
    bones

id0 = [digits.finger_id]==0;
nnz(id0)

ans = 140

bones = [digits.bones]

bones = 1x2800 struct array with fields:
    prev_joint
    next_joint
    width
    rotation

[bones.prev_joint]

ans = 1x8400

-116.7911 162.2039 -46.7425 -116.7911 162.2039 -46.7425 -119.9917 117.6783 -38.0319 -126.3976 96.3697 -35.3235 -108.4055 162.7472 -61.3988 -93.4886 98.3964 -77.4282 -92.6329 72.1629 -68.9765 -100.3854 67.2915 -57.0133 -113.6001 162.8399 -68.6146 -105.5847 102.6094 -90.1403

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Perhaps something like this:

thumb = [digits(id0).bones]

thumb = 1x560 struct array with fields:
    prev_joint
    next_joint
    width
    rotation

or perhaps this, which returns the first element of each structure array:

first = cellfun(@(s)s(1),{digits(id0).bones})

first = 1x140 struct array with fields:
    prev_joint
    next_joint
    width
    rotation

[first.prev_joint]

ans = 1x420

-116.7911 162.2039 -46.7425 -118.7844 162.6437 -45.3555 -121.9578 165.5228 -42.9163 -122.7603 168.2865 -42.1373 -55.5596 379.1909 -25.3262 -124.2343 170.6235 -41.0462 -56.3998 382.6714 -26.1379 -125.5027 172.1781 -40.9584 -57.1402 385.6229 -26.9821 -126.1411 173.0077 -41.2141

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Answer 2

[SAI SRUJAN]

Hi Julia,

I understand that you are facing an issue appending data when extracting nested fields as inidividual variables.

Please go through the following code sample to proceed further,

has_hands = ~cellfun(@isempty, {first100.hands});
filtered_first100 = first100(has_hands);
id = {filtered_first100.id};
time = {filtered_first100.timestamp};
hands = {filtered_first100.hands};

hand_type = {};

for i = 1:numel(hands)
    if isequal(size(hands{i}), [1,1])
        hand_type{end+1} = hands{i}.type;
    elseif isequal(size(hands{i}), [1,2])
        hand_type{end+1} = hands{i}(1).type;
        hand_type{end+1} = hands{i}(2).type;
    end
end

The variable 'hands' is a 1x99 struct, containing 41 structs within it, each of size [1,2]. Therefore, the expected size of 'hand_types' should be 140 double as we are parallelly storing the values instead of nesting.

I hope this helps!

[Stephen23]

"The variable 'hands' is a 1x99 struct"

No, it is actually a cell array (not a structure). Lets check it right now (note that MATLAB tells us it is a cell array too):

first100 = load('frames.mat').first100;
has_hands = ~cellfun(@isempty, {first100.hands});
filtered_first100 = first100(has_hands);
hands = {filtered_first100.hands} % <- here you define HANDS as a cell array.

hands = 1x99 cell array
  Columns 1 through 11

    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}

  Columns 12 through 22

    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}

  Columns 23 through 33

    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}

  Columns 34 through 44

    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}

  Columns 45 through 55

    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}

  Columns 56 through 66

    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}

  Columns 67 through 77

    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x2 struct}

  Columns 78 through 88

    {1x2 struct}    {1x2 struct}    {1x2 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x2 struct}    {1x2 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}

  Columns 89 through 99

    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}    {1x1 struct}

isstruct(hands) % is it a struct array? (hint: no)

ans = logical
   0

iscell(hands) %  it is a cell array (exactly as I wrote)

ans = logical
   1

If you had used square brackets it would be a structure array (and more efficient data design).

"containing 41 structs within it, each of size [1,2]."

Lets now check that misinformation as well:

C = cellfun(@size,hands,'uni',0);
unique(vertcat(C{:}),'rows')

ans = 2x2

1 1 1 2

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Some of the structures are scalar (i.e. 1x1), some have size 1x2 (just as the OP correctly stated).

"Therefore, the expected size of 'hand_types' should be 140 double as we are parallelly storing the values instead of nesting."

But it won't be, because you defined it to be a cell array, not a double array (you can confirm this by actually running your code):

hand_type = {}; % <- cell array

Note that storing lots of scalar numerics in a cell array is very inefficient data design and should be avoided.

[Julia]

It worked!! Thank you so much! Your help is really appreciated.

I see that I made a mistake when assigning an empty cell array to hand_type. Could you please help me to understand why hand_type = cell(size(hands)); was wrong, and what's the difference between it and hand_type = {};? Sorry I'm quite new to Matlab and may be asking silly questions.

Besides, may I ask a further question? If I also need to double the values in "id" and "time" when "hands" has size [1,2], so that the data in "hands" can still match the id and timestamp, is it possible to do?