Including all permutations when using data.table[,,by=…]

Question

I have a large data.table that I am collapsing to the month level using ,by.

There are 5 by vars, with # of levels: c(4,3,106,3,1380)

Answer 1

Make a cartesian join of the unique values, and use that to join back to your results

dat.keys <- dat[,CJ(g1=unique(g1), g2=unique(g2), g3=unique(g3))]
setkey(datCollapsed, g1, g2, g3)
nrow(datCollapsed[dat.keys])  # effectively a left join of datCollapsed onto dat.keys
# [1] 625

Note that the missing values are NA right now, but you can easily change that to 0s if you want.