Spark Streaming mapWithState类型不兼容

时间:2018-07-19 14:05:32

标签: java apache-spark spark-streaming

我是Spark世界的新手,我正在尝试使用Java API使用mapWithState进行一些有状态的转换。不幸的是,有一个关于类型不兼容的警报,我确实无法解决。这是所涉及的代码,并指出了错误。我已经找到了一个类似的话题,建议更改进口

Function3<String, Optional<LinkedList<String>>, State<LinkedList<String>>, Tuple2<String, LinkedList<String>>> mappingFunction = 
            new Function3<String, Optional<LinkedList<String>>, State<LinkedList<String>>, Tuple2<String, LinkedList<String>>>() {
        public Tuple2<String, LinkedList<String>> call(String key, Optional<LinkedList<String>> value, State<LinkedList<String>> state) {
            if(state.exists()) {
                value.get().addAll(state.get());
                LinkedList<String> newState = value.get();
                Tuple2<String, LinkedList<String>> tuple = new Tuple2<String, LinkedList<String>>(key, newState);
                return tuple;
            }
            else {
                LinkedList<String> initialState = new LinkedList<String>();
                state.update(initialState);
                return new Tuple2<String, LinkedList<String>>(key, initialState);
            }
        }
    };
    JavaMapWithStateDStream<String, LinkedList<String>, LinkedList<String>, Tuple2<String,LinkedList<String>>> mapWithStateDStream =
            pairedStream.mapWithState(StateSpec.function(mappingFunction));
}

The error is: "The method mapWithState(StateSpec<String,String,StateType,MappedType>) in the type JavaPairDStream<String,String> is not applicable for the arguments (StateSpec<String,LinkedList<String>,LinkedList<String>,Tuple2<String,LinkedList<String>>>)"

0 个答案:

没有答案