Kafka流自动提交偏移

时间:2020-04-13 09:15:24

标签: apache-kafka stream autocommit

这是我的问题: My Top包含一个联接,该联接在某些处理器之后,在某些其他处理器之前,当联接处理器之后的处理器中发生未捕获的异常时,整个流应用程序都将关闭! 当我重新启动应用程序时,处理正在开始加入节点,而不是开始节点? 为什么?

以下是我的上衣:

Topologies:
   Sub-topology: 0
    Source: KSTREAM-SOURCE-0000000000 (topics: [paymentReqTopic])
      --> assignId
    Processor: assignId (stores: [])
      --> qrCodeOrWithhold
      <-- KSTREAM-SOURCE-0000000000
    Processor: qrCodeOrWithhold (stores: [])
      --> qrCodeOrWithhold-predicate-0, qrCodeOrWithhold-predicate-1
      <-- assignId
    Processor: qrCodeOrWithhold-predicate-1 (stores: [])
      --> deduplicatePayment
      <-- qrCodeOrWithhold
    Processor: deduplicatePayment (stores: [paymentStore])
      --> KSTREAM-FILTER-0000000007
      <-- qrCodeOrWithhold-predicate-1
    Processor: KSTREAM-FILTER-0000000007 (stores: [])
      --> doPaymentWithProvider
      <-- deduplicatePayment
    Processor: doPaymentWithProvider (stores: [paymentStore])
      --> selectReqId
      <-- KSTREAM-FILTER-0000000007
    Processor: selectReqId (stores: [])
      --> KSTREAM-FILTER-0000000011
      <-- doPaymentWithProvider
    Processor: KSTREAM-FILTER-0000000011 (stores: [])
      --> KSTREAM-SINK-0000000010
      <-- selectReqId
    Sink: KSTREAM-SINK-0000000010 (topic: selectReqId-repartition)
      <-- KSTREAM-FILTER-0000000011
    Processor: qrCodeOrWithhold-predicate-0 (stores: [])
      --> none
      <-- qrCodeOrWithhold

  Sub-topology: 1
    Source: KSTREAM-SOURCE-0000000002 (topics: [paymentRespTopic])
      --> KSTREAM-WINDOWED-0000000014
    Source: KSTREAM-SOURCE-0000000012 (topics: [selectReqId-repartition])
      --> KSTREAM-WINDOWED-0000000013
    Processor: KSTREAM-WINDOWED-0000000013 (stores: [KSTREAM-JOINTHIS-0000000015-store])
      --> KSTREAM-JOINTHIS-0000000015
      <-- KSTREAM-SOURCE-0000000012
    Processor: KSTREAM-WINDOWED-0000000014 (stores: [KSTREAM-JOINOTHER-0000000016-store])
      --> KSTREAM-JOINOTHER-0000000016
      <-- KSTREAM-SOURCE-0000000002
    Processor: KSTREAM-JOINOTHER-0000000016 (stores: [KSTREAM-JOINTHIS-0000000015-store])
      --> KSTREAM-MERGE-0000000017
      <-- KSTREAM-WINDOWED-0000000014
    Processor: KSTREAM-JOINTHIS-0000000015 (stores: [KSTREAM-JOINOTHER-0000000016-store])
      --> KSTREAM-MERGE-0000000017
      <-- KSTREAM-WINDOWED-0000000013
    Processor: KSTREAM-MERGE-0000000017 (stores: [])
      --> KSTREAM-SINK-0000000018
      <-- KSTREAM-JOINTHIS-0000000015, KSTREAM-JOINOTHER-0000000016
    Sink: KSTREAM-SINK-0000000018 (topic: paymentTopic)
      <-- KSTREAM-MERGE-0000000017

以及make this comment

当我重新启动流,重做联接操作而不是最火的处理器时,发生异常并关闭我的流吗?为什么

0 个答案:

没有答案