Question

我正在尝试从Spark 1.6连接ElasticSerach 5.6.6。以下是在我的代码中配置的。还有什么我想念的吗？

  "xpack.security.user"
  "xpack.security.transport.ssl.enabled"
  "xpack.security.transport.ssl.keystore.path"
  "xpack.security.transport.ssl.keystore.password"
  "xpack.security.transport.ssl.truststore.path"
  "xpack.security.transport.ssl.truststore.password"

获取以下问题：

2018-03-07 07:33:01,509 WARN  [task-result-getter-2] scheduler.TaskSetManager(70): Lost task 2.0 in stage 6.0 (TID 150, ip-172-30-4-42.ec2.internal, executor 3): org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:250)
at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:546)
at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:58)
at org.elasticsearch.spark.rdd.EsSpark$$anonfun$doSaveToEs$1.apply(EsSpark.scala:102)
at org.elasticsearch.spark.rdd.EsSpark$$anonfun$doSaveToEs$1.apply(EsSpark.scala:102)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)

引起：org.elasticsearch.hadoop.rest.EsHadoopTransportException：javax.net.ssl.SSLException：java.lang.RuntimeException：意外错误：java.security.InvalidAlgorithmParameterException：trustAnchors参数必须为非空在org.elasticsearch.hadoop.rest.NetworkClient.execute（NetworkClient.java:124）在org.elasticsearch.hadoop.rest.RestClient.execute（RestClient.java:444）在org.elasticsearch.hadoop.rest.RestClient.execute（RestClient.java:424）在org.elasticsearch.hadoop.rest.RestClient.execute（RestClient.java:428）在org.elasticsearch.hadoop.rest.RestClient.get（RestClient.java:154） at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion（RestClient.java:609）在org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion（InitializationUtils.java:243）

Caused by: org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:124)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:444)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:424)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:428)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:154)
at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:609)
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:243)

Answer 1

当我们在spark文件路径中执行作业时，需要file:///<yourdirectory>/<filename>

Elasticsearch SSL trustAnchor问题

1 个答案: