Question

我正在使用aws sdk构建一个spark应用程序来访问S3源代码。我收到以下错误：

java.lang.NoSuchMethodError： org.apache.http.conn.ssl.SSLConnectionSocketFactory。（Ljavax /净/ SSL /的SSLContext; Ljavax /净/ SSL /的HostnameVerifier;）V

在线查找解决方案，看来我的spark应用程序使用了错误的httpclient。以下线程似乎提供了解决方案，但不确定如何覆盖默认的默认httpclient。

What version of httpclient is compatible with the Amazon SDK v 1.11.5?

以下是我系统中的不同httpclients。

./Applications/IBM Notes.app/Contents/MacOS/shared/eclipse/plugins/org.apache.wink_1.1.2.20150826-0855/lib/httpclient-4.0.1.jar
./Users/XXXXX/.ivy2/cache/org.apache.httpcomponents/httpclient/jars/httpclient-4.1.2.jar
./Users/XXXXX/.ivy2/cache/org.apache.httpcomponents/httpclient/jars/httpclient-4.5.1.jar
./Users/XXXXX/.m2/repository/org/apache/httpcomponents/httpclient/4.0.2/httpclient-4.0.2.jar
./Users/XXXXX/.m2/repository/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6.jar
./Users/XXXXX/Downloads/aws-java-sdk-1.11.110/third-party/lib/httpclient-4.5.2.jar
./usr/local/aws-java/aws-java-sdk-1.11.109/third-party/lib/httpclient-4.5.2.jar
./usr/local/spark/spark-2.1.0-bin-hadoop2.7/jars/httpclient-4.5.2.jar
./usr/local/zeppelin/interpreter/alluxio/httpclient-4.3.6.jar
./usr/local/zeppelin/interpreter/bqsql/httpclient-4.3.6.jar
./usr/local/zeppelin/interpreter/elasticsearch/httpclient-4.3.6.jar
./usr/local/zeppelin/interpreter/hbase/httpclient-4.3.6.jar
./usr/local/zeppelin/interpreter/kylin/httpclient-4.3.6.jar
./usr/local/zeppelin/interpreter/lens/httpclient-4.3.6.jar
./usr/local/zeppelin/interpreter/livy/httpclient-4.3.4.jar
./usr/local/zeppelin/interpreter/pig/httpclient-4.3.6.jar
./usr/local/zeppelin/lib/httpclient-4.3.6.jar
./usr/local/zeppelin/lib/interpreter/httpclient-4.3.6.jar

我没有指定classpath，所以我不确定它正在挑选哪个httpclient。我如何覆盖它总是选择./usr/local/aws-java/aws-java-sdk-1.11.109/third-party/lib/httpclient-4.5.2.jar？

Answer 1

将httpclient-4.5.2.jar和httpcore-4.4.4.jar复制到zeppelin / interpreter / spark文件夹中可以解决此错误。

覆盖httpclient版本

1 个答案: