Question

我正在通过围绕各种功能添加一些重试逻辑来修改现有的流图。其中之一就是来源，在这种情况下，它恰好是来自alpakka kafka连接器的kafka Consumer.committableSource。在下游，该图期望使用Source[ConsumerMessage.CommittableMessage[String, AnyRef], Control]类型，但是当我将可提交表源包装在RestartSource中时，我得到的是Source[ConsumerMessage.CommittableMessage[String, AnyRef], NotUsed]

我尝试在最后添加(Keep.both)，但最终出现编译时错误。这是两个示例供参考：

val restartSource: Source[ConsumerMessage.CommittableMessage[String, AnyRef], NotUsed] = RestartSource.onFailuresWithBackoff(
  minBackoff = 3.seconds,
  maxBackoff = 60.seconds,
  randomFactor = .2
) {() => Consumer.committableSource(consumerSettings, subscription)}

val s: Source[ConsumerMessage.CommittableMessage[String, AnyRef], Control] = Consumer.committableSource(consumerSettings, subscription)

Answer 1

正如您所观察到的那样，并且如当前打开的ticket所讨论的那样，原始Source的物化值未包含在包装RestartSource的返回值中。要解决此问题，请尝试使用mapMaterializedValue（免责声明：我没有测试以下内容）：

val restartSource: Source[ConsumerMessage.CommittableMessage[String, AnyRef], Control] = {
  var control: Option[Control] = None

  RestartSource.onFailuresWithBackoff(
    minBackoff = 3.seconds,
    maxBackoff = 60.seconds,
    randomFactor = .2
  ) { () =>
    Consumer.committableSource(consumerSettings, subscription)
      .mapMaterializedValue { c =>
        control = Some(c)
      }
  }
  .mapMaterializedValue(_ => control)
  .collect { case Some(c) => c }
}

Answer 2

您可以preMaterialize Source来产生Control，如下所示：

Pair<Consumer.Control, Source<ConsumerMessage.CommittableOffset, NotUsed>> controlSourcePair =
    origSrc.preMaterialize(materializer);

Source<ConsumerMessage.CommittableOffset, NotUsed> source =
    RestartSource.withBackoff(
        Duration.ofSeconds(1),
        Duration.ofSeconds(10),
        0.2,
        20,
        controlSourcePair::second);
source
    .toMat(Committer.sink(CommitterSettings.create(system)
        .withMaxBatch(1)), Keep.both())
    .mapMaterializedValue(pair -> 
        Consumer.createDrainingControl(
            new Pair<>(controlSourcePair.first(), pair.second())))
    .run(materializer);

不为您提供与Scala类似的功能的歉意。

RestartSource掩盖包装后的源的物化值吗？

2 个答案: