Kafka Streams App-计数和总计

时间:2019-02-28 15:49:29

标签: java apache-kafka apache-kafka-streams

我正在尝试从KGroupedStream创建一个KTable,以存储每个键的值之和。

 final StreamsBuilder builder = new StreamsBuilder();
 final KTable<String, Long> sum = builder.stream("streams-plaintext-input")
            .groupByKey()
            .aggregate(new Initializer<Long>() {
                @Override
                public Long apply() {
                    return Long.MIN_VALUE;
                }
            }, new Aggregator<String, Long, Long>() {
                @Override
                public Long apply(final String key, final Long value,final Long aggregate) {
                    aggregate += value;
                    return aggregate;
                }
            }, Materialized.<String, Long, KeyValueStore<Byte, byte[]>>as("counts-store"));

但出现错误:

The method aggregate(Initializer<VR>, Aggregator<? super Object,? super Object,VR>, Materialized<Object,VR,KeyValueStore<Bytes,byte[]>>) in the type KGroupedStream<Object,Object> is not applicable for the arguments (new Initializer<Long>(){}, new Aggregator<String,Long,Long>(){}, Materialized<String,Long,KeyValueStore<Byte,byte[]>>)

我看到的所有示例都将Serde作为第三个参数传递给我,但是我已经尝试过并得到一个非常相似的错误(我认为这可能来自较旧的版本,因为它与当前版本的签名不匹配实施?):

final StreamsBuilder builder = new StreamsBuilder();
    final KTable<String, Long> sum = builder.stream("streams-plaintext-input")
            .groupByKey()
            .aggregate(new Initializer<Long>() {
                @Override
                public Long apply() {
                    return Long.MIN_VALUE;
                }
            }, new Aggregator<String, Long, Long>() {
                @Override
                public Long apply(final String key, final Long value,final Long aggregate) {
                    aggregate += value;
                    return aggregate;
                }
            }, Serdes.Long());

错误:

The method aggregate(Initializer<VR>, Aggregator<? super Object,? super Object,VR>, Materialized<Object,VR,KeyValueStore<Bytes,byte[]>>) in the type KGroupedStream<Object,Object> is not applicable for the arguments (new Initializer<Long>(){}, new Aggregator<String,Long,Long>(){}, Serde<Long>)

我在做什么错了?

使用Kafka版本:2.1.0

1 个答案:

答案 0 :(得分:2)

您的代码中几乎没有问题:

  1. 对于{-# LANGUAGE Arrows, NoImplicitPrelude #-} module Main where import Prelude hiding ((.), id) import Control.Arrow import Control.Monad import Control.Category import Data.List import Data.Maybe data IOArrow a b = IOArrow { runIOArrow :: a -> IO b } instance Category IOArrow where id = IOArrow return IOArrow f . IOArrow g = IOArrow $ f <=< g instance Arrow IOArrow where arr f = IOArrow $ return . f first (IOArrow f) = IOArrow $ \(a, c) -> do x <- f a return (x, c) foo :: Int -> String foo = show bar :: String -> IO Int bar = return . read main :: IO () main = do let f = arr (++"!!") . arr foo . IOArrow bar . id result <- runIOArrow f "123" putStrLn result 而不是Materialized.as,您应该通过java.lang.Byte
  2. 您不应修改org.apache.kafka.common.utils.Bytes变量:final
  3. 您必须将键和值的类型添加到aggregate += value;调用(StreamsBuilder::stream

修改后,外观应大致如下:

builder.<String, Long>stream("streams-plaintext-input")