How do I use splitapply when the applied function has non-scalar output?

I am trying clean variables in a dataset, creating a new table variable, but splitapply does not seam to allow for nonscalar outputs of the function.

As a minimal working example, I have a table

myTable = table();
myTable.Gender = ["Male","Female","Male","Female"]';
myTable.Var1 = [1,2,3,4]';

And I am now trying to create a new table variable where the group mean is subtracted from "Var1" (yielding Var2 = [-1, -1, 1, 1])

When using splitapply(), I get the error

Error using splitapply (line 132)
The function '...' returned a non-scalar value when applied to the 1st group of data.

Is there a way to do this, also for function more involved than demeaning the input variables?

Thanks in advance,

Chris

Answer 1

[dpb]

OK, I reread the Q? and realize there was enough in it to see what it was that was wanted...

>> t=myTable;                       % shorter name; I'm lazy... :)
>> t.Gender=categorical(t.Gender);  % gender is a categorical variable; treat it as such
>> fnMN=@(v) {v-mean(v)};           % have to encapsulate the return values as cell
>> splitapply(fnMN,t.Var1,g)        % show what that returns is the cell array by group
ans =
  2×1 cell array
    {2×1 double}
    {2×1 double}
>> cell2mat(ans)    % and convert to the array form to see what values are...
ans =
    -1
     1
    -1
     1
>>

Now, the cell array is by group so to convert the cell array positioning to position in table :

grpmn=splitapply(fnMN,t.Var1,g);   % the group mean-adjusted values
for i=1:numel(grpmn)               % number of groups (2 here)
  t.Var2(i==g)=grpmn{i};           % spread cell values over rows in table
end

Above results in:

>> t
t =
  4×3 table
    Gender    Var1    Var2
    ______    ____    ____
    Male      1       -1  
    Female    2       -1  
    Male      3        1  
    Female    4        1  
>> 

[Walter Roberson]

Using a cell wrapper around the output should work. If you put the results into a table you could splitvars afterwards.

[dpb]

"Using a cell wrapper around the output should work."

Guess I don't follow your meaning here, Walter.

I don't yet have splitvars (R2017b still) but that would do it. I need to find the time to update...

[Walter Roberson]

I was agreeing with

fnMN=@(v) {v-mean(v)};

which puts a cell array around the result.

[dpb]

Ah! I was thinking you were also referring to a way to unwrap the output of the cell array...

[clauper]

That was what I was looking for!

There are two small bugs in your answer, though. if you use

grpmn=splitapply(fnMN,t.Var1,g);       % the group mean-adjusted values
for i=1:  numel(grpmn)                 % number of groups (2 here)
  t.Var2(i==g)=grpmn{i};               % spread cell values over rows in table
end

it works like a charm. Thanks a lot!

[dpb]

I see I inadvertentedly didn't edit out the cell2mat that I had used initially before it dawned on me the order wasn't correct that way. I originally posted an ERRATUM on that answer but had meeting in town at the time had to go to so didn't have time to fix it then...sorry.

Glad you caught it. Corrected rest of original...

[clauper]

Glad you corrected it!

As a final remark, we should loop over

for i = 1 :  numel(grpmn) 

since numel(g)==4 is not correct.

Thanks again!

[dpb]

Ah so! I had changed variable names in command window and didn't catch one in the forum code...thanks.

Answer 2

[dpb]

[A,B,C,...]=splitapply(func, ...);

applies func to inputs by group and returns the given outputs...

See the doc for details, examples.

The function uses the group inputs for each call and outputs M (scalar) variables. Each output can be a different type/class, but must be consistent from call to call for each output.

[clauper]

Dear dpb

Thanks for your answer.

Your solution allows for having multiple vectors as inputs. It still does not allow 'func' to have non-scalar output (same error).

Thanks anyways, Chris