Question

我试图使用以下代码基于键生成计数，该代码基于单词计数示例。奇怪的是，如果mapValues函数在String上返回，则groupBy会按注释行中所述工作，但是当我发送String的键对作为键，而GenericRecord的键对作为值时发送。

final Serde<String> stringSerde = Serdes.String();
final Serde<Long> longSerde = Serdes.Long();

final Map<String, String> serdeConfig = Collections.singletonMap("schema.registry.url","http://localhost:8081");

stringSerde.configure(serdeConfig, true); // `true` for record keys
final Serde<GenericRecord> valueGenericAvroSerde = new GenericAvroSerde();
                valueGenericAvroSerde.configure(serdeConfig, false); // `false` for record values

StreamsBuilder builder = new StreamsBuilder();
KStream<String, GenericRecord> textLines =
                  builder.stream("ora-query-in",Consumed.with(stringSerde, valueGenericAvroSerde));


final KTable<String, Long> wordCounts = textLines       
                        .mapValues(new ValueMapperWithKey<String, GenericRecord, KeyValue<String, GenericRecord>>() {

                                    @Override
                                    public KeyValue<String, GenericRecord> apply(String arg0, GenericRecord arg1) {

                                        return new KeyValue<String, GenericRecord>(arg1.get("KEY_FIELD").toString(),arg1);
                                        }
                                    })

            //                      .groupBy((key, value) -> value) //THIS WORKS if value is STRING
            //                      .groupBy((key, value) -> key) //DOES NOT WORK EITHER
                                    .groupByKey() //THIS does nothing
                                    .count();
wordCounts.toStream().to("test.topic.out",Produced.with(stringSerde, longSerde));

我是否缺少配置

streamsConfiguration.put(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://localhost:8081");

streamsConfiguration.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
streamsConfiguration.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());

Answer 1

您还没有写错什么，但是Serialization似乎是个问题

您可以使用：

KStream::groupBy(final KeyValueMapper<? super K, ? super V, KR> selector, final Grouped<KR, V> grouped)。

someStream.groupByKey((key, value) -> value, Grouped.with(newKeySerdes, valueSerdes)

KGroupedStream::count(final Materialized<K, Long, KeyValueStore<Bytes, byte[]>> materialized)

someGroupedStream.count(Materialized.with(newKeySerdes, valueSerdes)

可能是相同原因，例如：

Kafka流groupByKey不适用于count（）

1 个答案: