MATLAB Answers

Fill in missing NaNs

9 views (last 30 days)
Cuong Nguyen
Cuong Nguyen on 30 Mar 2020
Commented: Cuong Nguyen on 31 Mar 2020
I am trying to fill these NaNs following this rule: If there is a single NAN, I want the NAN to be filled in with the average of the numbers before and after. If there is more than one NAN. I want the NAN to be filled in with the nearest number. For example, row 2305 should be the average of 16.3 and 14.8, from 2298 to 2304 should be 16.3, and from 2306 to 2312 should be 14.8.

  4 Comments

Show 1 older comment
Cuong Nguyen
Cuong Nguyen on 30 Mar 2020
L1 = length(maxTemp);
for i = 1:1:L1
for j = 0:20
if i <27028 && isnan(maxTemp(i+j))
a=j;
else
break
end
end
if isnan(maxTemp(i)) && isnan (maxTemp(i+a)) % 'a' distance away from i
for k = 0:a % new variable which goes upto variable a
f = a; %new variable which goes down from variable a
if (a/2 > k)
maxTemp(i+k)=maxTemp(i+k-1);
maxTemp(i+f)=maxTemp(i+f+1);
else
break
end
if mod(a,2)== 0 % if a is even
maxTemp(i+k)= avg(maxTemp(i+k-1),maxTemp(i+k+1));
elseif mod(a,2)~= 0 % if a is not even
maxTemp(i+k)=maxTemp(i+k-1);
maxTemp(i+f)=maxTemp(i+f+1);
end
f = f-1; % decrease by 1
end
end
end
That array is showing maximum temperature day by day (on the photo is day 2297 to day 2313). After I run my code, this is what I got:
I need row 2305 to be average of 16.3 and 14.8, and row 2298 - 2304 to be 16.3. Thanks a lot for your time.
Image Analyst
Image Analyst on 30 Mar 2020
Why not use interp1() or regionfill() to linearly interpolate from one side to the other?
Cuong Nguyen
Cuong Nguyen on 30 Mar 2020
I am quite a newbie to matlab, can you make it more specific? Thanks.

Sign in to comment.

Accepted Answer

Ameer Hamza
Ameer Hamza on 30 Mar 2020
Edited: Ameer Hamza on 30 Mar 2020
Try this:
x = [1;2;3;4;nan;nan;nan;nan;nan;5;7;8;nan;nan;nan;nan;11;11;12;nan;nan;nan;15];
bb = regionprops(isnan(x));
idx_nnan = find(~isnan(x));
idx_nan = find(isnan(x));
[~,idx] = min(abs(idx_nan - idx_nnan'), [], 2);
x(idx_nan) = x(idx_nnan(idx));
a = [bb.Centroid];
a(1:2:end) = [];
bb = bb(a==fix(a));
for i=1:numel(bb)
idx_center = bb(i).Centroid(2);
idx = cumsum(bb(i).BoundingBox([2 4])) + [-0.5 0.5];
x(idx_center) = mean(x(idx));
end
Original x:
x =
1
2
3
4
NaN
NaN
NaN
NaN
NaN
5
7
8
NaN
NaN
NaN
NaN
11
11
12
NaN
NaN
NaN
15
New x:
x =
1.0000
2.0000
3.0000
4.0000
4.0000 % <--- nearesr value
4.0000 % <--- nearesr value
4.5000 % average
5.0000 % <--- nearesr value
5.0000 % <--- nearesr value
5.0000
7.0000
8.0000
8.0000 % <--- nearesr value
8.0000 % <--- nearesr value
11.0000 % <--- nearesr value
11.0000 % <--- nearesr value
11.0000
11.0000
12.0000
12.0000 % <--- nearesr value
13.5000 % average
15.0000 % <--- nearesr value
15.0000

  4 Comments

Show 1 older comment
Ameer Hamza
Ameer Hamza on 31 Mar 2020
Cuong, this code uses a tricky way to find the center point of each NaN cluster by using a function regionprops() from the image processing toolbox. This function is normally used to detect similar regions in an image and give bounding boxes around those regions. I applied that function on a 1D array to find clusters of NaN in your input array. The Centroid gives the center point of NaN, and BoundingBox gives the edges of each NaN cluster.
Now the line
[~,idx] = min(abs(idx_nan - idx_nnan'), [], 2);
is used to find all the NaN which are not at center and find their nearest non-NaN number.
Cuong Nguyen
Cuong Nguyen on 31 Mar 2020
Oh nice nice. Thank you, I got something to learn from this!
Ameer Hamza
Ameer Hamza on 31 Mar 2020
Glad to be of help.

Sign in to comment.

More Answers (2)

darova
darova on 30 Mar 2020
Use bwlabel
A1 = isnan(A); % find NaN
[L,n] = bwlabel(A1); % label each region
xx = 1:length(A);
for i = 1:n % loop through each region
BW = L==i; % select region
if sum(BW(:))>1 % if more than one 'NaN'
ix1 = find(BW,1,'first'); % first index of region
ix2 = floor(mean(BW.*xx)); % find mean index in region
ix3 = find(B2,1,'last'); % last index of region
A(ix1:ix2) = A(ix1-1); % fill first part
A(ix2+1:ix3) = A(ix3+1); % fill second part
else
ix = find(BW);
A(ix) = mean(A(ix([-1 1]))); % average
end
end

  1 Comment

Cuong Nguyen
Cuong Nguyen on 31 Mar 2020
I tried yours and I got this message "Array indices must be positive integers or logical values". It also shows that the error is in "A(ix) = mean(A(ix([-1 1])))" line.

Sign in to comment.


Andrei Bobrov
Andrei Bobrov on 31 Mar 2020
This question is a repeat of this question:
x = [1;2;3;4;nan;nan;nan;nan;nan;5;7;8;nan;nan;nan;nan;11;11;12;nan;nan;nan;15];
out = [x,f1(x)]
x = [16.3;nan(15,1);14.8];
out = [x, f1(x)]
Here f1:
function out = f1(x)
b1 = fillmissing(x,'linear');
b2 = fillmissing(x,'nearest');
d = [0;diff(bwdist(~isnan(x)),2);0]==-2;
out = b2;
out(d) = b1(d);
end

  3 Comments

Cuong Nguyen
Cuong Nguyen on 31 Mar 2020
Oh icic, and in your answer you said "In R2016b". Is there any feature in r2016b that other versions doesnt' have?
Andrei Bobrov
Andrei Bobrov on 31 Mar 2020
No! It only says that this version is available to me :) and this code will work with R2016b and later.
Cuong Nguyen
Cuong Nguyen on 31 Mar 2020
Ahh ok. Can you explain more about this line? It's something about the distance to the edges of NaN cluster I reckon, isn't it?
d = [0;diff(bwdist(~isnan(x)),2);0]==-2;

Sign in to comment.

Sign in to answer this question.

Tags


Translated by