Question

我有一个图表，从sqs读取，写入另一个系统，然后从sqs删除。为了从sqs中删除，我需要在SqsMessage对象上使用收据句柄

在Http流的情况下，流的签名允许我说出从流的下游发出哪种类型，

Flow[(HttpRequest, T), (Try[HttpResponse], T), HostConnectionPool]

在这种情况下，我可以将T设置为SqsMessage，我仍然拥有我需要的所有数据。

然而，一些连接器，例如google cloud pub sub，发出了一个完全没用的（对我而言）pub子ID。

pub sub子流的下游我需要能够访问pub子流之前的sqs消息id。

在不重写pub子连接器

的情况下解决此问题的最佳方法是什么

我在概念上想要这样的东西：

Flow[SqsMessage] //i have my data at this point
within(
.map(toPubSubMessage)
.via(pubSub))

... from here i have the same type i had before within however it still behaves like a linear graph with back pressure etc

Answer 1

您可以使用PassThrough集成模式。作为使用示例，请查看akka-streams-kafka - ＆gt;班级akka.kafka.scaladsl.Producer - ＆gt; Mehtod def f low[K, V, PassThrough]

所以，只需使用PassThrough元素实现您自己的舞台，例如akka.kafka.internal.ProducerStage[K, V, PassThrough]

package my.package

import java.util.concurrent.atomic.AtomicInteger

import scala.concurrent.Future
import scala.util.{Failure, Success, Try}

import akka.stream._
import akka.stream.ActorAttributes.SupervisionStrategy
import akka.stream.stage._

final case class Message[V, PassThrough](record: V, passThrough: PassThrough)

final case class Result[R, PassThrough](result: R, message: PassThrough)

class PathThroughStage[R, V, PassThrough]
  extends GraphStage[FlowShape[Message[V, PassThrough], Future[Result[R, PassThrough]]]] {

  private val in = Inlet[Message[V, PassThrough]]("messages")
  private val out = Outlet[Result[R, PassThrough]]("result")
  override val shape = FlowShape(in, out)

  override protected def createLogic(inheritedAttributes: Attributes) = {
    val logic = new GraphStageLogic(shape) with StageLogging {
      lazy val decider = inheritedAttributes.get[SupervisionStrategy]
        .map(_.decider)
        .getOrElse(Supervision.stoppingDecider)
      val awaitingConfirmation = new AtomicInteger(0)
      @volatile var inIsClosed = false

      var completionState: Option[Try[Unit]] = None

      override protected def logSource: Class[_] = classOf[PathThroughStage[R, V, PassThrough]]

      def checkForCompletion() = {
        if (isClosed(in) && awaitingConfirmation.get == 0) {
          completionState match {
            case Some(Success(_)) => completeStage()
            case Some(Failure(ex)) => failStage(ex)
            case None => failStage(new IllegalStateException("Stage completed, but there is no info about status"))
          }
        }
      }

      val checkForCompletionCB = getAsyncCallback[Unit] { _ =>
        checkForCompletion()
      }

      val failStageCb = getAsyncCallback[Throwable] { ex =>
        failStage(ex)
      }

      setHandler(out, new OutHandler {
        override def onPull() = {
          tryPull(in)
        }
      })

      setHandler(in, new InHandler {
        override def onPush() = {
          val msg = grab(in)
          val f = Future[Result[R, PassThrough]] {
            try {
              Result(// TODO YOUR logic
                msg.record,
                msg.passThrough)
            } catch {
              case exception: Exception =>
                decider(exception) match {
                  case Supervision.Stop =>
                    failStageCb.invoke(exception)
                  case _ =>
                    Result(exception, msg.passThrough)
                }
            }

            if (awaitingConfirmation.decrementAndGet() == 0 && inIsClosed) checkForCompletionCB.invoke(())
          }
          awaitingConfirmation.incrementAndGet()
          push(out, f)
        }

        override def onUpstreamFinish() = {
          inIsClosed = true
          completionState = Some(Success(()))
          checkForCompletion()
        }

        override def onUpstreamFailure(ex: Throwable) = {
          inIsClosed = true
          completionState = Some(Failure(ex))
          checkForCompletion()
        }
      })

      override def postStop() = {
        log.debug("Stage completed")
        super.postStop()
      }
    }
    logic
  }
}

Akka流改变第三方流程/阶段的返回类型

1 个答案: