Question

我正在尝试部署点火器，以便可以将共享的RDD / Dataframe缓存用于我的Spark集群。我已遵循spark安装说明，并选择将其部署到我现有的运行spark的纱线簇中。我正在使用HDP部署Spark。

我已经验证资源管理器和历史记录服务器正在侦听以下端口，并且可以远程登录到每个端口。我究竟做错了什么？我不是按照预期的方式部署这种东西吗？

我正在跑步：

yarn jar ignite-yarn-2.6.0.jar ./ignite-yarn-2.6.0.jar ../../../cluster.properties

以下错误：

18/09/24 22:13:38 INFO client.RMProxy: Connecting to ResourceManager at dev01clus02.dna.local/172.31.31.5:8050
18/09/24 22:13:38 INFO client.AHSProxy: Connecting to Application History server at dev01clus02.dna.local/172.31.31.5:10200
Exception in thread "main" java.lang.RuntimeException: Failed update ignite.
        at org.apache.ignite.yarn.IgniteProvider.updateIgnite(IgniteProvider.java:243)
        at org.apache.ignite.yarn.IgniteProvider.getIgnite(IgniteProvider.java:93)
        at org.apache.ignite.yarn.IgniteYarnClient.getIgnite(IgniteYarnClient.java:194)
        at org.apache.ignite.yarn.IgniteYarnClient.main(IgniteYarnClient.java:84)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:209)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:675)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
        at org.apache.ignite.yarn.IgniteProvider.updateIgnite(IgniteProvider.java:220)
        ... 9 more

Answer 1

似乎是GridGain为Apache Ignite项目提供的一部分基础结构目前无法正常工作。我会提出这个问题。

同时，您可以提供指向解压缩的Apache Ignite 2.6分发目录的IGNITE_PATH属性（在配置，系统属性或环境中），以避免完全尝试下载。

点燃纱线和Hortonworks

1 个答案: