You can use non-equi join in data.table
and then sum points
which overlap in the range.
library(data.table)
dt[out, .(start, end, game, player, points),
on=.(player, game>=start, game<=end)][
, .(points = sum(points)), by = .(start, end, player)]
# start end player points
#1: 1 2 ace 7
#2: 1 2 bob 8
#3: 3 3 casey 2