mapping a Stream with a function returning a Future

Question

I sometimes find myself in a situation where I have some Stream[X], and a function X => Future Y, that I\'d like to combine to a Future[Stream

Answer 1

Forgetting about Stream:

import scala.concurrent.Future
import ExecutionContext.Implicits.global

val x = 1 to 10 toList
def toFutureString(value : Int) = Future {
  println("starting " + value)
  Thread.sleep(1000)
  println("completed " + value)
  value.toString
}

yields (on my 8 core box):

scala> Future.traverse(x)(toFutureString)
starting 1
starting 2
starting 3
starting 4
starting 5
starting 6
starting 7
starting 8
res12: scala.concurrent.Future[List[String]] = scala.concurrent.impl.Promise$DefaultPromise@2d9472e2

scala> completed 1
completed 2
starting 9
starting 10
completed 3
completed 4
completed 5
completed 6
completed 7
completed 8
completed 9
completed 10

So 8 of them get kicked off immediately (one for each core, though that's configurable via the threadpool executor), and then as those complete more are kicked off. The Future[List[String]] returns immediately, and then after a pause it starts printing those "completed x" messages.

An example use of this could be when you have a List[Url's], and a function of type Url => Future[HttpResponseBody]. You could call Future.traverse on that list with that function, and kick off those http requests in parallel, getting back a single future that's a List of the results.

Was something that like what you were going for?