Question

我从akka http doc中看到这个简单的例子： http://doc.akka.io/docs/akka-http/current/scala/http/client-side/request-level.html

我稍微修改了一下，要求提出一百个请求。应用程序在32个请求（默认import $ from 'jquery'; import React from 'react'; import { connect } from 'react-redux'; import { FormattedMessage } from 'util/IntlComponents'; import OkeyMatchResult from './OkeyMatchResult'; import { RoomState } from 'constants/AppConstants'; function formattedMessage(message) { return <FormattedMessage message={message}/>; } // This should return functions which internally use the Redux dispatch method (the dispatch dependency will be injected by react-redux) function mapDispatchToProps (dispatch) { var rematchAction = {}; // Define your "rematch" Redux action here return { onRematch: function () => { dispatch(rematchAction); // This will dispatch the action to your Redux store } } }; // This creates an instance of OkeyMatchResultDialog with the object returned by mapDispatchToProps injected into its props export default connect(null, mapDispatchToProps)(OkeyMatchResultDialog); // This stays a purely presentational component (not aware of Redux) class OkeyMatchResultDialog extends React.Component { componentWillReceiveProps(nextProps) { const currentRoomState = this.props.roomState; const nextRoomState = nextProps.roomState; if (currentRoomState === RoomState.PLAYING && nextRoomState === RoomState.END) { $('#matchResultModal').openModal({ opacity: 0.5 }); } } render() { const matchEndTitle = formattedMessage('room_title.match_end'); const rematchTitle = formattedMessage('room_title.rematch'); const backToLobbyTitle = formattedMessage('room_title.back_tolobby'); const { matchResult } = this.props; let PlayAgainButton = ''; PlayAgainButton = ( <a onClick={this.props.onRematch} > {rematchTitle}</a> // Bound to the injected Redux-aware function ); return ( <div id='matchResultModal' className='matchresult-modal modal modal-fixed-footer'> <div className='modal-content'> <h4 className='center'>{matchEndTitle}</h4> <OkeyMatchResult matchResult={matchResult}/> </div> <div className='modal-footer'> <a className='btn side-by-side'>{backToLobbyTitle}</a> <a className='btn'>{PlayAgainButton}</a> <a className='btn'>{rematchTitle}</a> </div> </div> ); } }配置）之后阻塞。为什么呢？

max-open-requests

输出：

import akka.actor.{Actor, ActorLogging, ActorSystem, Props}
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.{ActorMaterializer, ActorMaterializerSettings}
import akka.util.ByteString

import scala.io.StdIn

object AkkaClientExample extends App {
  val system: ActorSystem = ActorSystem("BatchAkka")
  try {
    val unformattedAddresses = (1 to 100).map(i => s"Rue de la Gracieuse $i, Préverenges, Switzerland")

    val googleGeocoder = system.actorOf(GoogleGeocoder.props, "GoogleGeocoder")

    unformattedAddresses.foreach(e => googleGeocoder ! GoogleGeocoder.GeoCode(e))

    println(">>> Press ENTER to exit <<<")
    StdIn.readLine()
  } finally {
    system.terminate()
  }
}

object GoogleGeocoder {
  def props: Props = Props[GoogleGeocoder]

  final case class GeoCode(unformattedAddress: String)
}

class GoogleGeocoder extends Actor with ActorLogging {
  import GoogleGeocoder._
  import akka.pattern.pipe
  import context.dispatcher

  final implicit val materializer: ActorMaterializer = ActorMaterializer(ActorMaterializerSettings(context.system))

  val http = Http(context.system)  

  def receive = {
    case GeoCode(unformattedAddress) =>
      log.info(s"GeoCode $unformattedAddress")
      http
        .singleRequest(HttpRequest(uri = url(unformattedAddress)))
        .map(r => (unformattedAddress, r))
        .pipeTo(self)

    case (unformattedAddress: String, resp @ HttpResponse(StatusCodes.OK, headers, entity, _)) =>
      log.info(s"Success response comming for $unformattedAddress")
      entity.dataBytes.runFold(ByteString(""))(_ ++ _).foreach { body =>
        val response = body.utf8String.replaceAll("\\s+", " ").take(50)
        log.info(s"Success response for $unformattedAddress: $response")
      }

    case (unformattedAddress: String, resp @ HttpResponse(code, _, _, _)) =>
      log.info(s"Request failed, response code: $code for $unformattedAddress")
      resp.discardEntityBytes()
  }

  def url(unformattedAddress: String): String =
    //s"https://maps.googleapis.com/maps/api/geocode/json?address=${URLEncoder.encode(unformattedAddress, "UTF-8")}&key=${URLEncoder.encode(googleApiKey, "UTF-8")}"
    s"https://www.epfl.ch/"
}

在前32次请求后被阻止。

更新考虑到@ shutty的回答：

我已经按如下方式修改了程序，它可以工作：

[INFO] [07/28/2017 20:08:26.977] [BatchAkka-akka.actor.default-dispatcher-4] [akka://BatchAkka/user/GoogleGeocoder] GeoCode Rue de la Gracieuse 1, Préverenges, Switzerland
[INFO] [07/28/2017 20:08:27.080] [BatchAkka-akka.actor.default-dispatcher-4] [akka://BatchAkka/user/GoogleGeocoder] GeoCode Rue de la Gracieuse 2, Préverenges, Switzerland
...
[INFO] [07/28/2017 20:08:27.098] [BatchAkka-akka.actor.default-dispatcher-13] [akka://BatchAkka/user/GoogleGeocoder] GeoCode Rue de la Gracieuse 99, Préverenges, Switzerland
[INFO] [07/28/2017 20:08:27.098] [BatchAkka-akka.actor.default-dispatcher-13] [akka://BatchAkka/user/GoogleGeocoder] GeoCode Rue de la Gracieuse 100, Préverenges, Switzerland

[INFO] [07/28/2017 20:08:27.615] [BatchAkka-akka.actor.default-dispatcher-11] [akka://BatchAkka/user/GoogleGeocoder] Success response comming for Rue de la Gracieuse 1, Préverenges, Switzerland
[INFO] [07/28/2017 20:08:27.620] [BatchAkka-akka.actor.default-dispatcher-11] [akka://BatchAkka/user/GoogleGeocoder] Success response comming for Rue de la Gracieuse 4, Préverenges, Switzerland
[INFO] [07/28/2017 20:08:27.668] [BatchAkka-akka.actor.default-dispatcher-17] [akka://BatchAkka/user/GoogleGeocoder] Success response for Rue de la Gracieuse 4, Préverenges, Switzerland: <!doctype html><html lang="fr" class="no-js"><head
[INFO] [07/28/2017 20:08:27.668] [BatchAkka-akka.actor.default-dispatcher-21] [akka://BatchAkka/user/GoogleGeocoder] Success response for Rue de la Gracieuse 1, Préverenges, Switzerland: <!doctype html><html lang="fr" class="no-js"><head
...
[INFO] [07/28/2017 20:08:27.787] [BatchAkka-akka.actor.default-dispatcher-5] [akka://BatchAkka/user/GoogleGeocoder] Success response comming for Rue de la Gracieuse 31, Préverenges, Switzerland
[INFO] [07/28/2017 20:08:27.795] [BatchAkka-akka.actor.default-dispatcher-15] [akka://BatchAkka/user/GoogleGeocoder] Success response comming for Rue de la Gracieuse 32, Préverenges, Switzerland
[INFO] [07/28/2017 20:08:27.802] [BatchAkka-akka.actor.default-dispatcher-16] [akka://BatchAkka/user/GoogleGeocoder] Success response for Rue de la Gracieuse 31, Préverenges, Switzerland: <!doctype html><html lang="fr" class="no-js"><head
[INFO] [07/28/2017 20:08:27.806] [BatchAkka-akka.actor.default-dispatcher-17] [akka://BatchAkka/user/GoogleGeocoder] Success response for Rue de la Gracieuse 32, Préverenges, Switzerland: <!doctype html><html lang="fr" class="no-js"><head

所以，基本上是添加一个队列。

但是，有没有更好的方法来实现这一目标？

我想象这种实现可能失败的情况：例如，如果class GoogleGeocoder extends Actor with ActorLogging { import GoogleGeocoder._ import akka.pattern.pipe import context.dispatcher final implicit val materializer: ActorMaterializer = ActorMaterializer(ActorMaterializerSettings(context.system)) val http = Http(context.system) val queue = new scala.collection.mutable.Queue[String] var currentRequests = 0 val MaxCurrentRequest = 10 def receive = { case GeoCode(unformattedAddress) => if (currentRequests < MaxCurrentRequest) query(unformattedAddress) else queue += unformattedAddress case (unformattedAddress: String, resp @ HttpResponse(StatusCodes.OK, headers, entity, _)) => log.info(s"Success response comming for $unformattedAddress") entity.dataBytes.runFold(ByteString(""))(_ ++ _).foreach { body => currentRequests = currentRequests - 1 queryNext() val response = body.utf8String.replaceAll("\\s+", " ").take(50) log.info(s"Success response for $unformattedAddress: $response") } case (unformattedAddress: String, resp @ HttpResponse(code, _, _, _)) => log.info(s"Request failed, response code: $code for $unformattedAddress") resp.discardEntityBytes() currentRequests = currentRequests - 1 queryNext() case f: Status.Failure => log.info("failure" + textSample(f)) case m => log.info("unexpected message: " + textSample(m)) } def query(unformattedAddress: String) { log.info(s"GeoCode $unformattedAddress") http .singleRequest(HttpRequest(uri = url(unformattedAddress))) .map(r => (unformattedAddress, r)) .pipeTo(self) } def queryNext() { if (queue.nonEmpty) { query(queue.dequeue) } } def url(unformattedAddress: String): String = //s"https://maps.googleapis.com/maps/api/geocode/json?address=${URLEncoder.encode(unformattedAddress, "UTF-8")}&key=${URLEncoder.encode(googleApiKey, "UTF-8")}" s"https://www.epfl.ch/" }产生失败的未来，http.singleRequest将不会减少。我可以在currentRequests处理此问题，但是，此解决方案看起来非常容易出错。

也许akka已经提供了一些处理队列的机制？

是否有办法向客户端添加反压（以便case f: Status.Failure：AkkaClientExample在达到unformattedAddresses.foreach(e => googleGeocoder ! GoogleGeocoder.GeoCode(e))时被阻止？）

Answer 1

如果您使用akka.logging = DEBUG运行示例，则会注意到以下输出：

InputBuffer (max-open-requests = 32) now filled with 31 request after enqueuing GET / Empty InputBuffer (max-open-requests = 32) now filled with 32 request after enqueuing GET / Empty InputBuffer (max-open-requests = 32) exhausted when trying to enqueue GET / Empty InputBuffer (max-open-requests = 32) exhausted when trying to enqueue GET / Empty InputBuffer (max-open-requests = 32) exhausted when trying to enqueue GET / Empty

akka-http如何处理客户端请求的汇集非常a comprehensive description，但简而言之，如果您使用更多的max-open-requests重载池，它将开始丢弃请求：

http
 .singleRequest(HttpRequest(uri = url(unformattedAddress)))
 .map(r => (unformattedAddress, r)) // <- HERE
 .pipeTo(self)

当您在Scala中对Future进行映射时，它将仅在成功的Future完成时执行您的回调，这在您的代码中不是这种情况。如果您以不同的方式重写代码，如：

http
  .singleRequest(HttpRequest(uri = url(unformattedAddress)))
  .onComplete {
    case Success(r) =>
      self ! (unformattedAddress, r)
    case Failure(ex) =>
      log.error(ex, "pool overflow")
  }

你会看到许多例外抱怨失败的未来。

更新：

至于我自己的观点，当你需要背压时，演员和溪流并不适合。作为一个选项，您可以完全重写代码而无需演员：

def url(addr: String) = "http://httpbin.org/headers"
implicit val system: ActorSystem = ActorSystem("BatchAkka")
implicit val mat: ActorMaterializer = ActorMaterializer()
import system.dispatcher
val http = Http()
val addresses = (1 to 100).map(i => s"Rue de la Gracieuse $i, Préverenges, Switzerland")
Source(addresses)
  .mapAsync(4)(addr => http.singleRequest(HttpRequest(uri = url(addr))))
  .map(response => println(response.status))
  .runWith(Sink.seq)
  .map(_ => println("done"))

在此解决方案中，您只需向服务器发出4个并行请求，包括背压，铃声和口哨声。

在max-open-requests之后阻止了akka客户端http

1 个答案: