Handling org.apache.arrow.vector.util.OversizedAllocationException error

Question

I have a spark data frame that has two columns, probability and true label (from a binary classifier). The data is already provided to me (so I am not performing any Mlib operat