data.table is slow compared to data.frame for group filtering

Question

data.table is slow if I filter based on numeric value (greater than or less than)

library(data.table)
df <- data.frame(group = sample(paste("group",