Question

我已经开始使用Akka Streams和Op-Rabbit而且有点困惑。

我需要根据谓词拆分流，然后将它们组合起来，就像我在创建图表和使用分区和合并时所做的那样。

我已经能够使用GraphDSL.Builder执行此类操作，但似乎无法使用AckedSource / Flow / Sink

图表看起来像：

                        | --> flow1 --> |
source--> partition --> |               | --> flow3 --> sink
                        | --> flow2 --> |

我不确定splitWhen是否应该使用，因为我总是需要2个流量。

这是一个不进行分区但不使用GraphDSL.Builder的示例：

def splitExample(source: AckedSource[String, SubscriptionRef],
                 queueName: String)
                (implicit actorSystem: ActorSystem): RunnableGraph[SubscriptionRef] = {
  val toStringFlow: Flow[AckTup[Message], AckTup[String], NotUsed] = Flow[AckTup[Message]]
    .map[AckTup[String]](tup => {
      val (p,m) = tup
      (p, new String(m.data))
    })

  val printFlow1: Flow[AckTup[String], AckTup[String], NotUsed] = Flow[AckTup[String]]
    .map[AckTup[String]](tup => {
      val (p, s) = tup
      println(s"flow1 processing $s")
      tup
     })

  val printFlow2: Flow[AckTup[String], AckTup[String], NotUsed] = Flow[AckTup[String]]
    .map[AckTup[String]](tup => {
      val (p, s) = tup
      println(s"flow2 processing $s")
      tup
    })

  source
    .map(Message.queue(_, queueName))
    .via(AckedFlow(toStringFlow))
    // partition if string.length < 10
    .via(AckedFlow(printFlow1))
    .via(AckedFlow(printFlow2))
    .to(AckedSink.ack)
}

这是我似乎无法正常工作的代码：

import GraphDSL.Implicits._
def buildModelAcked(source: AckedSource[String, SubscriptionRef] , queueName: String)(implicit actorSystem: ActorSystem):  Graph[ClosedShape, Future[Done]] = {
    import GraphDSL.Implicits._
    GraphDSL.create(Sink.ignore) { implicit builder: GraphDSL.Builder[Future[Done]] => s =>
    import GraphDSL.Implicits._
    source.map(Message.queue(_, queueName)) ~> AckedFlow(toStringFlow) ~> AckedSink.ack
//      source.map(Message.queue(_, queueName)).via(AckedFlow(toStringFlow)).to(AckedSink.ack)
    ClosedShape

}}

编译器无法解析~>运算符

所以我的问题是：

是否有使用scala dsl构建Acked / Source / Flow / Sink图形的示例项目？
是否有一个分区和合并的示例项目，与我在这里尝试的类似？

Answer 1

在处理acked-stream时，请记住以下定义。

AckedSource[Out, Mat]是Source[AckTup[Out], Mat]]
AckedFlow[In, Out, Mat]是Flow[AckTup[In], AckTup[Out], Mat]
AckedSink[In, Mat]是Sink[AckTup[In], Mat]
AckTup[T]是(Promise[Unit], T)
经典流程组合器将在T

AckTup

.acked组合子将完成Promise[Unit]的{{1}}

GraphDSL边缘运算符（AckedFlow）将对一堆Akka预定义形状起作用（请参阅GraphDSL.Implicits的代码），但它不会对acked定义的自定义形状起作用-stream lib。

你有两种出路：

您定义了自己的~>隐式运算符，与~>
你解开了阶段以获得标准阶段。您可以使用GraphDSL.Implicits访问已包装的舞台 - 可在.wrappedRepr，AckedSource和AckedFlow上找到。

Answer 2

基于Stefano Bonetti的出色方向，这是一个可能的解决方案：

graph:    
                        |--> short --|
  rabbitMq --> before --|            |--> after
                        |--> long  --|

溶液：

val before: Flow[AckTup[Message], AckTup[String], NotUsed] = Flow[AckTup[Message]].map[AckTup[String]](tup => {
  val (p,m) = tup
  (p, new String(m.data))
})

val short: Flow[AckTup[String], AckTup[String], NotUsed] = Flow[AckTup[String]].map[AckTup[String]](tup => {
  val (p, s) = tup
  println(s"short: $s")
  tup
})
val long: Flow[AckTup[String], AckTup[String], NotUsed] = Flow[AckTup[String]].map[AckTup[String]](tup => {
  val (p, s) = tup
  println(s"long: $s")
  tup
})
val after: Flow[AckTup[String], AckTup[String], NotUsed] = Flow[AckTup[String]].map[AckTup[String]](tup => {
  val (p, s) = tup
  println(s"all $s")
  tup
})

def buildSplitGraph(source: AckedSource[String, SubscriptionRef]
                    , queueName: String
                    , splitLength: Int)(implicit actorSystem: ActorSystem):  Graph[ClosedShape, Future[Done]] = {
 GraphDSL.create(Sink.ignore) { implicit builder: GraphDSL.Builder[Future[Done]] => s =>
   val toShort = 0
   val toLong = 1

   // junctions
   val split = builder.add(Partition[AckTup[String]](2, (tup: AckTup[String]) => {
                                                           val (p, s) = tup
                                                           if (s.length < splitLength) toShort else toLong
                                                         }
   ))
   val merge = builder.add(Merge[AckTup[String]](2))

   //graph
   val beforeSplit = source.map(Message.queue(_, queueName)).wrappedRepr ~> AckedFlow(before).wrappedRepr
   beforeSplit ~> split
   // must do short, then long since the split goes in that order
   split ~> AckedFlow(short).wrappedRepr ~> merge
   split ~> AckedFlow(long).wrappedRepr ~> merge
   // after the last AckedFlow, be sure to '.acked' so that the message will be removed from the queue
   merge ~> AckedFlow(after).acked ~> s

  ClosedShape
}}

正如Stefano Bonetti所说，关键是要使用与.wrappedRepr相关联的AckedFlow，然后使用.acked组合作为最后一步。

Akka Streams scala DSL和Op-Rabbit

2 个答案: