Eliminate tables with more than 15% NaN data

Hey all,

I have C.mat which has 125 tables. I want to remove tables if any of tmax_m or tmin_m or rrr24 or tmin_m columns have more than 15% NaN cells.

C.mat is attached.

Thank you all

Answer 1

[Adam Danz]

This cellfun is a bit obnoxious but it produces an output select that is a logical vector the same size as C where true values identify cells that will be eliminated due to having at least 1 column listed in cols that has more than maxNaN percent NaN values.

load('C.mat')
cols ={'tmax_m', 'in_m' 'rrr24' 'tmin_m'};
maxNan = 0.15; 

select = cellfun(@(T)any(mean(isnan(T{:,ismember(T.Properties.VariableNames,cols)}),1) > maxNan), C); 
C(select) = []; % eliminate selected cells

[BN]

Both codes are accurate, I found this one more understandable for me. Thank you very much, Walter Roberson and Adam Danz

Answer 2

[Walter Roberson]

filetables = load('C.mat');
numtables = numel(filetables);
tablenames = fieldnames(filetables);

droptable = false(1, numtables);

for K = 1 : numtables
    thistablename = tablenames{K};
    thistable = filetables.(thistablename);
    if mean(isnan(thistable.tmax_m)) > 0.15 || ...
       mean(isnan(thistable.tmin_m)) > 0.15 || ...
       mean(isnan(thistable.rrr24)) > 0.15
       droptable(K) = true;
    end
end

tables_to_drop = tablenames(droptable);
filetables = rmfield(filetables, tables_to_drop);

save('C_filtered.mat', '-struct', 'filetables');