How do I find the indices of NaN values in a cell array?

Hi, I have a 3390 x 1 cell array containing 4 x 30 doubles. Some of these doubles contain NaN values which I would like to replace by the preceding double. How do I detect the indices of the doubles containing 'NaN' values and then replace them with the preceding 4x30 double? I have attached the cell array in question. Thanks in advance.

Answer 1

[Star Strider]

Try this:

D = load('XTrain2_all.mat');
XTrain2_all = D.XTrain2_all;

hasNaN = cellfun(@nnz,cellfun(@isnan, XTrain2_all, 'Unif',0), 'Unif',0);    % Cells With ‘NaN’ Values
idx = find([hasNaN{:}]);                                                    % Their Indices
XTrain2_all(idx) = XTrain2_all(idx-1);                                      % Replace With Previous

hasNaN = cellfun(@nnz,cellfun(@isnan, XTrain2_all, 'Unif',0), 'Unif',0);    % Check
Check = find([hasNaN{:}]);                                                  % Check

.

[Walter Roberson]

Is it possible for there to be two nan blocks in a row? If so then more work would have to be done.

[Star Strider]

The NaN entries are (4x30) matrices, all elements of which are NaN. Replacing the NaN matrices with the matrix before it (as my code does here) solves the problem presented using the approach requested.

I did not detect any other matrices with any NaN values other than those that were completely NaN matrices.

[Walter Roberson]

The question is whether the task has the possibility that at some point, there will be two blocks in a row that have nan. If it can happen, then using mass assignment idx from idx-1 will not work. However, using a forward loop would work in such a case.

[Star Strider]

The indices (4) that I discovered are not consecutive. I checked. That possibility does not exist.

I also considered fillmissing, however since all the 120 elements in the NaN matrices are NaN, fillmissing is not applicable here.

[Walter Roberson]

My answer shows how to use fillmissing for this: fillmissing on the index vector

[Star Strider]

This (slightly edited) version of my original code works as requested:

D = load('XTrain2_all.mat');
XTrain2_all = D.XTrain2_all;

hasNaN = cellfun(@nnz,cellfun(@isnan, XTrain2_all, 'Unif',0), 'Unif',0);            % Cells With ‘NaN’ Values
idx = find([hasNaN{:}]);                                                            % Their Indices
CheckNumNaNs = cellfun(@nnz, cellfun(@isnan,XTrain2_all(idx), 'Unif',0), 'Unif',0)  % Check Number Of  NaN( Elements (Not Necessary For The REst Of The Code)
XTrain2_all(idx) = XTrain2_all(idx-1);                                              % Replace With Previous

hasNaN = cellfun(@nnz,cellfun(@isnan, XTrain2_all, 'Unif',0), 'Unif',0);            % Check
Check = find([hasNaN{:}]);                                                          % Check

The only addition is ‘CheckNumNaNs’, not required for the code, only demonstrating that all the elements of the matrices with NaN are completely NaN. The last two lines (commented as ‘% Check’) demonstrate that the result is as requested, and that none of the matrices in the resulting array have NaN elements.

[Cai Chin]

Hi, thank you very much for your suggestions. It is a possibility that there will be consecutive missing blocks, so is there any way of essentially scanning through the indices until the last non-NaN index is found and then replacing all the missing blocks with this?

Also, if there are multiple missing blocks from the first inde onwards, is it possible to scan forwards and replace the missing indices with the next non-NaN index values?

[Star Strider]

My pleasure!

‘It is a possibility that there will be consecutive missing blocks, so is there any way of essentially scanning through the indices until the last non-NaN index is found and then replacing all the missing blocks with this?’

That would likely require looping through the ‘idx’ values. The first ‘idx’ value would be replaced with the matrix preceeding it, and the subsequent ‘idx’ values as well. It could end up that for consecutive ‘idx’ values, the same matrix could be duplicated consecutively as the result.

‘Also, if there are multiple missing blocks from the first inde onwards, is it possible to scan forwards and replace the missing indices with the next non-NaN index values?’

That would likely require indexing in reverse, starting with the first full (non-NaN) cell matrix and going backwards. I have no idea how you would want to treat the rest of the array, whether going backwards from the last ‘idx’ value to the first would work, or if you would want to treat the various segments of the cell array differently.

In any event, all those possibilities would likely require a loop of some sort.

.

Answer 2

[Walter Roberson]

The below accounts for the possibility of multiple nan blocks.

It does not, however, account for the possibility that the first block is nan (there is no previous block to fill from in that case.)

hasnan = cellfun(@(C) any(isnan(C(:))), XTrain2_all);
idx = 1 : length(C);
idx(hasnan) = 1;
idx = fillmissing(idx, 'previous');
newC = C(idx);

[Walter Roberson]

Corrected (tested)

load XTrain2_all.mat
hasnan = cellfun(@(C) any(isnan(C(:))), XTrain2_all);
idx = 1 : length(XTrain2_all);
idx(hasnan) = nan;
idx = fillmissing(idx, 'previous');
newXTrain2_all = XTrain2_all(idx);

[Walter Roberson]

To also account for the possibility of the first block being nan:

load XTrain2_all.mat
hasnan = cellfun(@(C) any(isnan(C(:))), XTrain2_all);
idx = 1 : length(XTrain2_all);
idx(hasnan) = nan;
idx = fillmissing(fillmissing(idx, 'previous'),'next');
newXTrain2_all = XTrain2_all(idx);