Where to put dispatch.Http.shutdown() in case of cascading Http calls

问题

At Where to put dispatch.Http.shutdown() I asked where to place the call to dispatch.Http.shutdown() if there are n independent Http calls. Those n independent calls, however, are all on the same "level".

How about cascading Http calls, whereas outer1 and outer2 are independent of each other (like in my former question), the inner calls, however, depend on the result of the respective outer call.

val outer1 = dispatch.Http(... request1 ...)
val outer2 = dispatch.Http(... request2 ...)

outer1 onComplete {
  case Success(h) =>
    /*inner11*/ dispatch.Http(... request11 based on `h` ...) onComplete { ... }
    /*inner12*/ dispatch.Http(... request12 based on `h` ...) onComplete { ... }
  case Failure(e) => logger.debug(s"Error: $e")
}
outer2 onComplete {
  case Success(j) => 
    /*inner21*/ dispatch.Http(... request21 based on `j` ...) onComplete { ... }
    /*inner22*/ dispatch.Http(... request22 based on `j` ...) onComplete { ... }
  case Failure(e) => logger.debug(s"Error: $e")
}

dispatch.Future.sequence(outer1 :: outer2 :: Nil) onComplete { 
  case _ => dispatch.Http.shutdown() // <<<<< too early
}

Thanks, /nm

Update 1: Thanks to Kevin Wright, things are getting more clear. In the following I try to clarify why I need this cascading futures and the onComplete on the nested ones. Let's assume I want to get a list of URLs of all accessible GitHub repositories for an authenticated user:

object Main extends App with Logging {
  import scala.concurrent.ExecutionContext.Implicits.global

  def logFailure[T](f: Future[T]) =
    f onFailure { case x => logger.debug("Error: " + x.toString) }

  // Personal access token 
  val pat = "..."
  // GitHub API
  val github: Req = host("api.github.com").secure 
  // Http executor
  //val http = Http()

  // User profile of authenticated user
  val ur: Req = github / "user" <:< Map("Authorization" -> s"token $pat")

  // Retrieve *all* accessible repositories for authenticated user 
  Http(ur OK as.json4s.Json) flatMap { u =>
    // organizations' repos
    val inner1 = (u \ "organizations_url").toOption collect { case JString(org) =>
      Http(url(org) OK as.json4s.Json) flatMap { o =>
        (o \ "repos_url").toOption collect { case JString(rep) => 
          Http(url(rep) OK as.json4s.Json) map { _ \ "html_url" }
        }
      }
    }
    // user's repos
    val inner2 = (u \ "repos_url").toOption collect { case JString(usr) => 
      Http(url(usr) OK as.json4s.Json) map { _ \ "html_url" }
    }

    ???
  }
}

Then as soon as I have retrieved all the URLs I want to spawn a git clone process for each.

Unfortunately inner1 does not yet type check. And still, if inner1: Option[Future[JValue]] and inner2: Option[Future[JValue]], then Future.sequence(inner1 :: inner2 :: Nil) does not type check.

回答1:

You'd find things easier if you separated the success and failure paths.

import dispatch.Future

def logFailure[T](f: Future[T]) =
  f onFailure { case x => logger.debug("Error: " + x.toString) }

val outer1 = dispatch.Http(... request1 ...) flatMap { h =>
  val in1 = dispatch.Http(... request11 based on `h` ...)
  val in2 = dispatch.Http(... request12 based on `h` ...)
  in1 onComplete { ... }
  in2 onComplete { ... }
  Future.sequence(in1:: in2 :: Nil)            
}

val outer2 = dispatch.Http(... request2 ...) flatMap { j =>
  val in1 = dispatch.Http(... request21 based on `j` ...)
  val in2 = dispatch.Http(... request22 based on `j` ...)
  in1 onComplete { ... }
  in2 onComplete { ... }
  Future.sequence(in1:: in2 :: Nil)            
}

logFailure(outer1)
logFailure(outer2)

Future.sequence(outer1 :: outer2 :: Nil) onComplete { 
  case _ => dispatch.Http.shutdown() // <<<<< too early
}

If not for the onComplete of those nested operations, it could almost certainly be done far more and with less duplication using a couple of comprehensions. As it is, I have no idea what you want to do inside of them.

来源：https://stackoverflow.com/questions/21791580/where-to-put-dispatch-http-shutdown-in-case-of-cascading-http-calls

标签

scala

future

scala-dispatch