MATLAB Answers

Eliminate tables with more than 15% NaN data

2 views (last 30 days)
Behzad Navidi
Behzad Navidi on 18 Jan 2020
Commented: Behzad Navidi on 18 Jan 2020
Hey all,
I have C.mat which has 125 tables. I want to remove tables if any of tmax_m or tmin_m or rrr24 or tmin_m columns have more than 15% NaN cells.
C.mat is attached.
Thank you all


Sign in to comment.

Accepted Answer

Adam Danz
Adam Danz on 18 Jan 2020
Edited: Adam Danz on 18 Jan 2020
This cellfun is a bit obnoxious but it produces an output select that is a logical vector the same size as C where true values identify cells that will be eliminated due to having at least 1 column listed in cols that has more than maxNaN percent NaN values.
cols ={'tmax_m', 'in_m' 'rrr24' 'tmin_m'};
maxNan = 0.15;
select = cellfun(@(T)any(mean(isnan(T{:,ismember(T.Properties.VariableNames,cols)}),1) > maxNan), C);
C(select) = []; % eliminate selected cells

  1 Comment

Behzad Navidi
Behzad Navidi on 18 Jan 2020
Both codes are accurate, I found this one more understandable for me. Thank you very much, Walter Roberson and Adam Danz

Sign in to comment.

More Answers (1)

Walter Roberson
Walter Roberson on 18 Jan 2020
filetables = load('C.mat');
numtables = numel(filetables);
tablenames = fieldnames(filetables);
droptable = false(1, numtables);
for K = 1 : numtables
thistablename = tablenames{K};
thistable = filetables.(thistablename);
if mean(isnan(thistable.tmax_m)) > 0.15 || ...
mean(isnan(thistable.tmin_m)) > 0.15 || ...
mean(isnan(thistable.rrr24)) > 0.15
droptable(K) = true;
tables_to_drop = tablenames(droptable);
filetables = rmfield(filetables, tables_to_drop);
save('C_filtered.mat', '-struct', 'filetables');


Sign in to comment.

Sign in to answer this question.





Translated by