Question

我有一个方法：def sightings(from: YearMonth): Future[Seq[Sighting]]

另一个：def sightings(from: YearMonth, to: YearMonth): Source[Sighting, NotUsed]

我想从YearMonth开始，以from开头，为每个to调用第一个，然后合并/连接结果。我似乎无法在Source上找到合适的方法来做到这一点。我现在的情况如下：

val months = from.until(to, ChronoUnit.MONTHS) + 1
Source.fromIterator(() => Iterator.range(0, months.toInt))
    .map(from.plusMonths(_))
    .mapAsyncUnordered(1)(sightings)

这会产生一个Source[Int, NotUsed]，而不是像我正在寻找的Source[Sighting, NotUsed]。

修改：我最终得到了以下内容：

trait Client {
  def sightings(yearMonth: YearMonth): Source[(HttpResponse, YearMonth), NotUsed]
}

trait HttpClient extends Client {
  implicit def system: ActorSystem

  private val yearMonthFormatter = DateTimeFormatter.ofPattern("yyyyMM")

  override def sightings(yearMonth: YearMonth): Source[(HttpResponse, YearMonth), NotUsed] = {
    Source.fromGraph(GraphDSL.create() { implicit b: GraphDSL.Builder[NotUsed] =>
      import GraphDSL.Implicits._

      // prepare graph elements
      val uri = "whatever.html"
      val src = Source.single(RequestBuilding.Get(uri))
      lazy val conn = Http().outgoingConnection("www.doesnotexist.com")
        .map((_, yearMonth))

      val flow = b.add(conn)

      // connect the graph
      src ~> flow

      // expose port
      SourceShape(flow.out)
    })
  }
}

trait Crawler {
  self: Client =>

  implicit def executionContext: ExecutionContext

  implicit def materializer: Materializer

  final def sightings(from: YearMonth, to: YearMonth): Source[Sighting, NotUsed] = {
    val months = from.until(to, ChronoUnit.MONTHS) + 1

    Source.fromIterator(() => Iterator.range(0, months.toInt))
      .map(x => from.plusMonths(x.toLong))
      .flatMapConcat(self.sightings)
      .mapAsyncUnordered(1)(t => {
        val (response, yearMonth) = (t._1, t._2)
        val body = Unmarshal(response.entity).to[String]
        val status = response.status

        responseMapper(body, yearMonth)(status)
      })
      .mapConcat(_.to[collection.immutable.Seq])
  }

  import scala.collection.JavaConversions._

  private def responseMapper(body: Future[String], yearMonth: YearMonth):
  PartialFunction[StatusCode, Future[Seq[Sighting]]] = {...}
}

Answer 1

根据第一个sightings函数（def sightings(from: YearMonth): Future[Seq[Sighting]]）的类型签名，听起来你想要一个这种类型的函数：

def fToS[A, B](f: A => Future[Seq[T]]): Flow[A, B, NotUsed]

然后你可以这样做：

val yearMonths: Source[YearMonth, NotUsed] = ??? // whatever you want
val toSightings: Flow[YearMonth, Sighting, NotUsed] = fToS(sightings)
val source: Source[Sighting, NotUsed] = yearMonths.via(toSightings)

fToS看起来像这样：

def fToS[A, B](f: A => Future[Seq[T]]): Flow[A, B, NotUsed] =
  Flow[A].mapAsync(1)(f).mapConcat(identity)

这应该有效。

Akka Streams：如何将未来[Seq [T]]转换为Source [T，NotUsed]

1 个答案: