Question

我有一些对性能不敏感的代码，并试图通过使用更少的期货来使堆栈更容易理解。这导致一些代码类似于以下内容：

  val fut = Future {
    val r = Future.traverse(ips) { ip =>
      val httpResponse: Future[HttpResponse] = asyncHttpClient.exec(req)
      httpResponse.andThen {
        case x => logger.info(s"received response here: $x")
      }
      httpResponse.map(r => (ip, r))
    }
    r.andThen { case x => logger.info(s"final result: $x") }
    Await.result(r, 10 seconds)
  }
  fut.andThen { x => logger.info(s"finished $x") }
  logger.info("here nonblocking")

正如预期的那样，http客户端中的内部日志记录显示响应立即返回，但执行logger.info(s"received response here: $x")和logger.info(s"final result: $x")的回调在Await.result(r, 10 seconds)次超时后才会执行。查看包含线程ID的日志输出，回调正在等待结果的同一线程（ForkJoinPool-1-worker-3）中执行，从而产生死锁。据我所知，ExecutionContext.global会在线程耗尽时按需创建额外的线程。这不是这种情况吗？似乎只有来自全局fork连接池的两个线程在日志（1和3）中产生任何输出。谁能解释一下呢？

至于修复，我知道也许最好的方法是将阻塞和非阻塞工作分成不同的线程池，但我希望通过使用动态大小的线程池来避免这种额外的簿记。有更好的解决方案吗？

Answer 1

如果要在线程被阻止时（暂时）增长池，请使用concurrent.blocking。在这里，你已经使用了所有的线程，进行了i / o，然后使用map和andThen（你不使用的结果）安排了更多的工作。

更多信息：您的“最终结果”预计会在遍历后执行，这是正常的。

阻止示例，尽管必须有SO Q＆amp; A：

scala> import concurrent._ ; import ExecutionContext.Implicits._

scala> val is = 1 to 100 toList
scala> def db = s"${Thread.currentThread}"
db: String

scala> def f(i: Int) = Future { println(db) ; Thread.sleep(1000L) ; 2 * i }
f: (i: Int)scala.concurrent.Future[Int]

scala> Future.traverse(is)(f _)
Thread[ForkJoinPool-1-worker-13,5,main]
Thread[ForkJoinPool-1-worker-7,5,main]
Thread[ForkJoinPool-1-worker-9,5,main]
Thread[ForkJoinPool-1-worker-3,5,main]
Thread[ForkJoinPool-1-worker-5,5,main]
Thread[ForkJoinPool-1-worker-1,5,main]
Thread[ForkJoinPool-1-worker-15,5,main]
Thread[ForkJoinPool-1-worker-11,5,main]
res0: scala.concurrent.Future[List[Int]] = scala.concurrent.impl.Promise$DefaultPromise@3a4b0e5d
[etc, N at a time]

与过度平行：

scala> def f(i: Int) = Future { blocking { println(db) ; Thread.sleep(1000L) ; 2 * i }}
f: (i: Int)scala.concurrent.Future[Int]

scala> Future.traverse(is)(f _)
Thread[ForkJoinPool-1-worker-13,5,main]
Thread[ForkJoinPool-1-worker-3,5,main]
Thread[ForkJoinPool-1-worker-1,5,main]
res1: scala.concurrent.Future[List[Int]] = scala.concurrent.impl.Promise$DefaultPromise@759d81f3
Thread[ForkJoinPool-1-worker-7,5,main]
Thread[ForkJoinPool-1-worker-25,5,main]
Thread[ForkJoinPool-1-worker-29,5,main]
Thread[ForkJoinPool-1-worker-19,5,main]

scala> Thread[ForkJoinPool-1-worker-23,5,main]
Thread[ForkJoinPool-1-worker-27,5,main]
Thread[ForkJoinPool-1-worker-21,5,main]
Thread[ForkJoinPool-1-worker-31,5,main]
Thread[ForkJoinPool-1-worker-17,5,main]
Thread[ForkJoinPool-1-worker-49,5,main]
Thread[ForkJoinPool-1-worker-45,5,main]
Thread[ForkJoinPool-1-worker-59,5,main]
Thread[ForkJoinPool-1-worker-43,5,main]
Thread[ForkJoinPool-1-worker-57,5,main]
Thread[ForkJoinPool-1-worker-37,5,main]
Thread[ForkJoinPool-1-worker-51,5,main]
Thread[ForkJoinPool-1-worker-35,5,main]
Thread[ForkJoinPool-1-worker-53,5,main]
Thread[ForkJoinPool-1-worker-63,5,main]
Thread[ForkJoinPool-1-worker-47,5,main]

ForkJoinPool只被两个工人阻止了

1 个答案: