Question

我正在编写一个kafka流2.3.0应用程序，以计算会话窗口中的事件数，并希望在会话超时时仅打印最终记录。

Serde<String> stringSerde = Serdes.serdeFrom(new StringSerializer(), new StringDeserializer());
Serde<MuseObject> museObjectSerde = Serdes.serdeFrom(new MuseObjectSerializer(), new MuseObjectDeserializer());
StreamsBuilder builder = new StreamsBuilder();
builder
  .stream(INPUT_TOPIC, Consumed.with(stringSerde, museObjectSerde))
  .map((key, value) -> {
    return KeyValue.pair(value.getSourceValue("vid"), value.toString());
  })
  .groupByKey(Grouped.with(Serdes.String(), Serdes.String())) 
  .windowedBy(SessionWindows.with(Duration.ofSeconds(INACTIVITY_GAP)).grace(Duration.ZERO))
  .count(Materialized.with(Serdes.String(), Serdes.Long()))
  .suppress(Suppressed.untilWindowCloses(Suppressed.BufferConfig.unbounded()))            
  .toStream()
  .print(Printed.toSysOut());

但是，会话超时时应用程序崩溃：

12:35:03.859 [kafka-producer-network-thread | kafka-streams-test-kgu-4c3f2398-8f67-429d-82ce-6062c86af466-StreamThread-1-producer] ERROR o.a.k.s.p.i.RecordCollectorImpl - task [1_0] Error sending record to topic kafka-streams-test-kgu-KTABLE-SUPPRESS-STATE-STORE-0000000008-changelog due to The server experienced an unexpected error when processing the request.; No more records will be sent and no more offsets will be recorded for this task. Enable TRACE logging to view failed record key and value.
org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
12:35:03.862 [kafka-streams-test-kgu-4c3f2398-8f67-429d-82ce-6062c86af466-StreamThread-1] ERROR o.a.k.s.p.i.AssignedStreamsTasks - stream-thread [kafka-streams-test-kgu-4c3f2398-8f67-429d-82ce-6062c86af466-StreamThread-1] Failed to commit stream task 1_0 due to the following error:
org.apache.kafka.streams.errors.StreamsException: task [1_0] Abort sending since an error caught with a previous record (key user01\x00\x00\x01m!\xCE\x99u\x00\x00\x01m!\xCE\x80\xD1 value null timestamp null) to topic kafka-streams-test-kgu-KTABLE-SUPPRESS-STATE-STORE-0000000008-changelog due to org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
    at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.recordSendError(RecordCollectorImpl.java:138)

我尝试注释掉“ .suppress ...”行。如果没有抑制（），它可以正常工作，并打印出类似这样的内容

[KSTREAM-FILTER-0000000011]: [user01@1568230244561/1568230250869], MuseSession{vid='user01', es='txnSuccess', count=6, start=2019-06-26 17:11:02.937, end=2019-06-26 18:07:10.685, sessionType='open'}".

在使用prevent（）时我错过了什么？有没有另一种方法可以仅过滤出已超时的会话记录？

感谢您的帮助。预先感谢。

Answer 1

suppress()至少需要代理版本0.11.0和消息格式0.11。

为什么启用了prevent（）的Kafka Stream应用程序崩溃？

1 个答案: