I have a 2d array (doubles
) representing some data, and it has a bunch of NaNs
in it. The contour plot of the data looks like this:
The technique I ended up using was the function nanconv.m at Matlab's File Exchange. It does exactly what I was looking for: it runs the filter in a way that ignores the NaNs
just the way that Matlab's built-in function nanmean
does. This is a hard to decipher from the documentation of the function, which is a tad cryptic.
Here's how I use it:
filtWidth = 7;
filtSigma = 5;
imageFilter=fspecial('gaussian',filtWidth,filtSigma);
dataFiltered = nanconv(data,imageFilter, 'nanout');
I'm pasting the nanconv
function below (it is covered by the BSD license). I will post images etc when I get a chance, just wanted to post what I ended up doing for anyone curious about what I did.
Comparison to other answers
Using gnovice's solution the results look intuitively very nice, but there are some quantitative blips on the edges that were a concern. In practice, the extrapolation of the image beyond the edges led to many spuriously high values at the edges of my data.
Using krisdestruction's suggestion of replacing the missing bits with the original data, also looks pretty decent (especially for very small filters), but (by design) you end up with unfiltered data at the edges, which is a problem for my application.
nanconv
function c = nanconv(a, k, varargin)
% NANCONV Convolution in 1D or 2D ignoring NaNs.
% C = NANCONV(A, K) convolves A and K, correcting for any NaN values
% in the input vector A. The result is the same size as A (as though you
% called 'conv' or 'conv2' with the 'same' shape).
%
% C = NANCONV(A, K, 'param1', 'param2', ...) specifies one or more of the following:
% 'edge' - Apply edge correction to the output.
% 'noedge' - Do not apply edge correction to the output (default).
% 'nanout' - The result C should have NaNs in the same places as A.
% 'nonanout' - The result C should have ignored NaNs removed (default).
% Even with this option, C will have NaN values where the
% number of consecutive NaNs is too large to ignore.
% '2d' - Treat the input vectors as 2D matrices (default).
% '1d' - Treat the input vectors as 1D vectors.
% This option only matters if 'a' or 'k' is a row vector,
% and the other is a column vector. Otherwise, this
% option has no effect.
%
% NANCONV works by running 'conv2' either two or three times. The first
% time is run on the original input signals A and K, except all the
% NaN values in A are replaced with zeros. The 'same' input argument is
% used so the output is the same size as A. The second convolution is
% done between a matrix the same size as A, except with zeros wherever
% there is a NaN value in A, and ones everywhere else. The output from
% the first convolution is normalized by the output from the second
% convolution. This corrects for missing (NaN) values in A, but it has
% the side effect of correcting for edge effects due to the assumption of
% zero padding during convolution. When the optional 'noedge' parameter
% is included, the convolution is run a third time, this time on a matrix
% of all ones the same size as A. The output from this third convolution
% is used to restore the edge effects. The 'noedge' parameter is enabled
% by default so that the output from 'nanconv' is identical to the output
% from 'conv2' when the input argument A has no NaN values.
%
% See also conv, conv2
%
% AUTHOR: Benjamin Kraus (bkraus@bu.edu, ben@benkraus.com)
% Copyright (c) 2013, Benjamin Kraus
% $Id: nanconv.m 4861 2013-05-27 03:16:22Z bkraus $
% Process input arguments
for arg = 1:nargin-2
switch lower(varargin{arg})
case 'edge'; edge = true; % Apply edge correction
case 'noedge'; edge = false; % Do not apply edge correction
case {'same','full','valid'}; shape = varargin{arg}; % Specify shape
case 'nanout'; nanout = true; % Include original NaNs in the output.
case 'nonanout'; nanout = false; % Do not include NaNs in the output.
case {'2d','is2d'}; is1D = false; % Treat the input as 2D
case {'1d','is1d'}; is1D = true; % Treat the input as 1D
end
end
% Apply default options when necessary.
if(exist('edge','var')~=1); edge = false; end
if(exist('nanout','var')~=1); nanout = false; end
if(exist('is1D','var')~=1); is1D = false; end
if(exist('shape','var')~=1); shape = 'same';
elseif(~strcmp(shape,'same'))
error([mfilename ':NotImplemented'],'Shape ''%s'' not implemented',shape);
end
% Get the size of 'a' for use later.
sza = size(a);
% If 1D, then convert them both to columns.
% This modification only matters if 'a' or 'k' is a row vector, and the
% other is a column vector. Otherwise, this argument has no effect.
if(is1D);
if(~isvector(a) || ~isvector(k))
error('MATLAB:conv:AorBNotVector','A and B must be vectors.');
end
a = a(:); k = k(:);
end
% Flat function for comparison.
o = ones(size(a));
% Flat function with NaNs for comparison.
on = ones(size(a));
% Find all the NaNs in the input.
n = isnan(a);
% Replace NaNs with zero, both in 'a' and 'on'.
a(n) = 0;
on(n) = 0;
% Check that the filter does not have NaNs.
if(any(isnan(k)));
error([mfilename ':NaNinFilter'],'Filter (k) contains NaN values.');
end
% Calculate what a 'flat' function looks like after convolution.
if(any(n(:)) || edge)
flat = conv2(on,k,shape);
else flat = o;
end
% The line above will automatically include a correction for edge effects,
% so remove that correction if the user does not want it.
if(any(n(:)) && ~edge); flat = flat./conv2(o,k,shape); end
% Do the actual convolution
c = conv2(a,k,shape)./flat;
% If requested, replace output values with NaNs corresponding to input.
if(nanout); c(n) = NaN; end
% If 1D, convert back to the original shape.
if(is1D && sza(1) == 1); c = c.'; end
end