尝试按照Flink文档(https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/elasticsearch.html)中给出的示例向Flink Elasticsearch接收器添加异常处理程序。
为Scala提供的示例代码比Scala具有更多的Java语言,请复制粘贴到此处:
val input: DataStream[String] = ...
input.addSink(new ElasticsearchSink(
config, transportAddresses,
new ElasticsearchSinkFunction[String] {...},
new ActionRequestFailureHandler {
@throws(classOf[Throwable])
override def onFailure(ActionRequest action,
Throwable failure,
int restStatusCode,
RequestIndexer indexer) {
if (ExceptionUtils.containsThrowable(failure,
EsRejectedExecutionException.class)) {
// full queue; re-add document for indexing
indexer.add(action)
} else if (ExceptionUtils.containsThrowable(failure,
ElasticsearchParseException.class)) {
// malformed document; simply drop request without failing
sink
} else {
// for all other failures, fail the sink
// here the failure is simply rethrown, but users can also
choose to throw custom exceptions
throw failure
}
}
}))
这是我的版本:
import org.apache.flink.util.{Collector, ExceptionUtils}
import org.elasticsearch.common.util.concurrent.EsRejectedExecutionException
// create elastic search sink
val esSinkBuilder = new ElasticsearchSink.Builder[DataStream](
httpHosts,
new ESSinkFunctionDataAggr(aggr_key, es_index, es_mapping),
new ActionRequestFailureHandler {
@throws(classOf[Throwable])
override def onFailure(action: ActionRequest,
failure: Throwable,
restStatusCode: Int,
indexer: RequestIndexer) {
if (ExceptionUtils.containsThrowable(failure, classOf[EsRejectedExecutionException])) {
// full queue; re-add document for indexing
indexer.add(action)
} else if (ExceptionUtils.containsThrowable(failure, classOf[ElasticsearchParseException])) {
// malformed document; simply drop request without failing sink
} else {
// for all other failures, fail the sink
// here the failure is simply rethrown, but users can also choose to throw custom exceptions
throw failure
}
}
}
)
该代码无法编译,IntelliJ用“无法解析符号containsThrowable” 的错误将其涂红。不确定如何解决此问题。我已经看到了另一个示例(Flink - ElasticSearch Sink - error handling),该示例使用findThrowableWithMessage而不是containsThrowable,但是就我的情况而言,我想按Flink文档中的说明处理异常。