I've done this with Clojure, which proved pretty effective for the following reasons:
- Being on the JVM is a huge advantage in terms of libraries. This effectively ruled out Haskell and Ocaml for my purposes, as we needed easy access to the Java ecosystem and integration with JVM based tools (Maven build etc.)
- You can drop into pure Java if you need to tightly optimise inner loops. We did this for some custom code processing large double[] arrays, but 99% of the time Clojure can get you the performance you need. See http://www.infoq.com/presentations/Why-Prismatic-Goes-Faster-With-Clojure for some examples of how to make Clojure go really fast (quite technical video, assumes some prior knowledge!). Once you start counting the ease of exploiting multiple cores, Clojure is very competitive on performance.
- Clojure has very nice multi-core concurrency support. This proved extremely useful for managing concurrent tasks. See http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey
- The REPL makes a very good environment for testing and exploratory work on data.
- Clojure is lazy which makes it suitable for handling larger-than-memory data sets (assuming you are careful not to try and force the whole data set into memory at once). There are also some nice libraries available in such an environment, most notable are Storm and Aleph. Storm may be particularly interesting for you, as it's designed for distributed realtime processing of large numbers of events.
I can't speak with quite so much experience of the other languages, but my impression from some practical experience of Haskell and Scala is:
- Haskell is great if you care about purity and strict functional programming with static types. The static typing can be a strong guarantee of correctness so might make this suitable for highly algorithmic work. Personally, I find pure FP a little too rigid - there are many times when mutable state is useful and I think Clojure has a slightly better balance here (by allowing controlled muability thorugh managed references).
- Scala is a great language and shares with Clojure the advantages of being on the JVM. To me Scala is more like a "better Java" with functional features and a very impressive type system. It's less of a paradigm shift from Clojure. Downside is that the type system can get quite complex / confusing.
Overall, I think you could be happy with any of these. It will probably come down to how much you care about the JVM and your view on type systems.
