弹性搜索水槽可分割性

Question

是否可以序列化ElasticSearch接收器并使用Stream

传递它

我有类似

的东西

stream.addSink(new ElasticsearchSink<>(config, transports, new IndexRequestBuilder<ObjectNode>() {
           ...

}
return eventStream;

但是它说Apache Flink中的ElasticsearchSink不可序列化。

Answer 1

我遇到了同样的问题并且发现如果将ClosureCleaner.ensureSerializable()定义为内部类，则Flink的IndexRequestBuilder方法会引发序列化错误：

org.apache.flink.api.common.InvalidProgramException: Object org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSink@62d24761 not serializable
...
Caused by: java.io.NotSerializableException: org.apache.flink.streaming.api.environment.LocalStreamEnvironment

如果您在自己的班级中实施IndexRequestBuilder，那么一切都应该有效：

stream.addSink(new ElasticsearchSink<>(
    config, transports, new MyIndexRequestBuilder<>()));