Finding closest point from other data frame

Question

I have two data frames , one is with 0.8 million rows with x and Y coordinates, another data frame is of 70000 rows with X and Y coordinates. I want to know logic and code in R

Answer 1

I found a faster way to get the expected result using the data.table library:

library(data.table)

time0 <- Sys.time()

Here is some random data:

df1 <- data.table(x = runif(8e5), y = runif(8e5))
df2 <- data.table(x = runif(7e4), y = runif(7e4))

Assuming (x,y) are the coordinates in an orthonormal coordinate system, you can compute the square of the distance as follow:

dist <- function(a, b){
                dt <- data.table((df2$x-a)^2+(df2$y-b)^2)
                return(which.min(dt$V1))}

And now you can applied this function to your data to get the expected result:

results <- df1[, j = list(Closest =  dist(x, y)), by = 1:nrow(df1)]

time1 <- Sys.time()
print(time1 - time0)

It tooked me around 30 minutes to get the result on a slow computer.

EDIT:

As asked, I have tried severals other solutions using sapply or using adply from the plyr package. I have tested these solutions on smaller data frames to make it faster.

library(data.table)
library(plyr)
library(microbenchmark)

########################
## Test 1: data.table ##
########################

dt1 <- data.table(x = runif(1e4), y = runif(1e4))
dt2 <- data.table(x = runif(5e3), y = runif(5e3))

dist1 <- function(a, b){
                dt <- data.table((dt2$x-a)^2+(dt2$y-b)^2)
                return(which.min(dt$V1))}

results1 <- function() return(dt1[, j = list(Closest =  dist1(x, y)), by = 1:nrow(dt1)])

###################
## Test 2: adply ##
###################

df1 <- data.frame(x = runif(1e4), y = runif(1e4))
df2 <- data.frame(x = runif(5e3), y = runif(5e3))

dist2 <- function(df){
                dt <- data.table((df2$x-df$x)^2+(df2$y-df$y)^2)
                return(which.min(dt$V1))}

results2 <- function() return(adply(.data = df1, .margins = 1, .fun = dist2))

####################
## Test 3: sapply ##
####################

df1 <- data.frame(x = runif(1e4), y = runif(1e4))
df2 <- data.frame(x = runif(5e3), y = runif(5e3))

dist2 <- function(df){
                dt <- data.table((df2$x-df$x)^2+(df2$y-df$y)^2)
                return(which.min(dt$V1))}

results3 <- function() return(sapply(1:nrow(df1), function(x) return(dist2(df1[x,]))))

microbenchmark(results1(), results2(), results3(), times = 20)

#Unit: seconds
#       expr      min       lq     mean   median       uq      max neval
# results1() 4.046063 4.117177 4.401397 4.218234 4.538186 5.724824    20
# results2() 5.503518 5.679844 5.992497 5.886135 6.041192 7.283477    20
# results3() 4.718865 4.883286 5.131345 4.949300 5.231807 6.262914    20

The first solution seems to be significantly faster than the 2 other. This is even more true for a larger dataset.