It's very difficult to answer questions like this without trying. So why don't you
- Figure out what would be a difficult operation
- Implement that (and I mean the simplest, quickest hack that you can make work)
- Run it with a lot of data, and see how long it takes
- Figure out if it's too slow
I've done this in the past and it's really the way to see if something performs well enough for something.