I have a table with missing values and I'm trying to write a function that will replace the missing values with a calculation based on the nearest two non-zero values.
Example:
X Tom 1 4.3 2 5.1 3 NA 4 NA 5 7.4 For X = 3, Tom = 5.1 + (7.4-5.1)/2.
For X = 4, Tom = (5.1 + (7.4-5.1)/2) + (7.4-5.1)/2
Does this function already exist? If not, any advice would be greatly appreciated.
A more usual way to do this (but not equivalent to the question) is to use linear interpolation:
library(zoo) df <- data.frame(X = 1:5, Tom = c(4.3, 5.1, NA, NA, 7.4)) na.approx(df) or spline interpolation:
na.spline(df) Just use a loop in this scenario, other approaches are much harder.
for (i in seq_len(nrow(df)) { if (is.na(df[i, 'Tom'])) df[i, 'Tom'] <- ((tmp <- c(0, df$Tom[!is.na(df$Tom)], 0))[i+1] + tmp[i]) / 2 + tmp[i] } df <- data.frame(X = seq_len(100), Tom = ifelse(runif(100, 0, 1) > 0.5, NA, round(runif(100, 0, 10), 1))) head(df) # X Tom # 1 1 NA # 2 1.4 # 3 3 NA # 4 4 3.9 # 5 5 NA for (i in seq_len(nrow(df))) { if (is.na(df[i, 'Tom'])) df[i, 'Tom'] <- ((tmp <- c(0, df$Tom[!is.na(df$Tom)], 0))[i+1] + tmp[i]) / 2 + tmp[i] } head(df) # X Tom # 1 1 0.70 # 2 2 1.40 # 3 3 4.05 # 4 4 3.90 # 5 5 9.05 Actually the imputeTS package (I am the maintainer) offers a good solutions for this.
Replacement with the Moving Average
na.ma(x, k = 2) x is your input object k is the moving average window
k of 1 means you only consider the values before and after k of 2 means you consider the 2 values before and the 2 values after
This function is probably the closest to the required calculation. The only difference is, that the imputeTS method does not jump over NA values. (as required by the thread starter)
But especially for long NA streaks this makes perfectly sense. 1, 2, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 14, 15, 16 (taking the average of 2 and 14 for the NA at position 3 would be no good idea)
Furthermore Last Observation Forward (as mentioned by 42 in the comment)
imputeTS::na.locf(x) or Interpolation (as also mentioned by G. Grothendieck)
imputeTS::na.interpolation(x) are also missing data replacement options that go a little bit in the same direction. Here is a introduction to the imputeTS package in the R Journal if you are interested.