Can't merge tables with error of weird array size limitation, help?

3 vues (au cours des 30 derniers jours)
Konstantin Maier
Konstantin Maier le 15 Mai 2021
I got two tables, one with training data and one with created features that I want to implement.
Table A has the size 307511x241 and is like so:
>> head(app_train, 3)
ans =
3×241 table
SK_ID_CURR NAME_CONTRACT_TYPE CODE_GENDER_M CODE_GENDER_F FLAG_OWN_CAR etc.
__________ __________________ _____________ _____________ ____________
100002 0 1 0 0 etc.
100003 0 0 1 0 etc.
100004 1 1 0 1 etc.
Table B has the size 307511x36 and is like so:
>> head(polyFeaturesTrain, 3)
ans =
3×36 table
Polynomial constant EXT_SOURCE_1 EXT_SOURCE_1^2 EXT_SOURCE_1^3 etc.
___________________ __________________ ___________________ __________________
1 0.0830369673913226 0.00689513795354757 0.0005725513454074 etc.
1 0.311267311381222 0.0968873391344949 0.0301578615592749 etc.
1 0.505997930505754 0.256033905676106 0.129552626411415 etc.
5 variables in Table B are the same as in Table A (EXT_SOURCE_1, EXT_SOURCE_2, EXT_SOURCE_3, DAYS_BIRTH and TARGET). The values for TARGET are the same in both tables, while for the other variables there are some missing values in Table A but not in Table B.
I want to merge Table B into Table A (rows stay the same, only add the variables) and for identical variables want to keep the values from Table B. I tried innerjoin, outerjoin, join and some different Name-Value Pair Arguments for those functions, nothing works. Also trying TableC = [TableA, TableB] doesn't work because of the identical variables.
When using the function I often get this error message:
Error using tabular/outerjoin (line 228)
Requested 1x80527655221 (600.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become
unresponsive.
The table I want would be 307511x272, how can it be 600GB? I don't understand what Matlab is trying to tell me.
If someone knows what I am doing wrong, please help me out, I spent the last few hours making no progress because of this.

Réponse acceptée

Stephen23
Stephen23 le 15 Mai 2021
"I want to merge Table B into Table A (rows stay the same, only add the variables) and for identical variables want to keep the values from Table B"
x = randi(9,7,1);
y = randi(9,7,1);
z = randi(9,7,1);
A = table(x,y)
A = 7×2 table
x y _ _ 4 6 1 7 1 9 8 1 1 3 2 5 8 7
y = round(y/2);
B = table(y,z)
B = 7×2 table
y z _ _ 3 4 4 5 5 4 1 4 2 2 3 4 4 3
V = setdiff(A.Properties.VariableNames,B.Properties.VariableNames);
C = [A(:,V),B]
C = 7×3 table
x y z _ _ _ 4 3 4 1 4 5 1 5 4 8 1 4 1 2 2 2 3 4 8 4 3

Plus de réponses (0)

Catégories

En savoir plus sur Tables dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by