Question

嗨，我刚刚配置了Spark群集模式，我正在尝试启动pyspark shell，但出现错误。我的集群中有2个奴隶和1个主人。我的配置运行良好，我的主人和工人在命令行jps中显示，并且在我的浏览器中也显示http://localhost:8080。但是我的问题是，当我尝试进入pyspark shell时，我得到了以下日志：

**18/07/05 10:30:53 WARN StandaloneAppClient$ClientEndpoint: Failed to 
connect to master XXX.XX.XX.80:7077**
org.apache.spark.SparkException: Exception thrown in awaitResult: 
at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:205)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:100)
at org.apache.spark.rpc.RpcEnv.setupEndpointRef(RpcEnv.scala:108)
atorg.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint$$ano nfun$tryRegisterAllMasters$1$$anon$1.run(StandaloneAppClient.scala:106)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
**Caused by: java.io.IOException: Failed to connect to /XXX.XX.XX.80:7077**
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:232)
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:182)
at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:197)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:194)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:190)
... 4 more
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: 
**Connexion refusée: /XXX.XX.XX.XX:7077**
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:257)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:291)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:631)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:566)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
... 1 more

PS：拒绝连接表示连接被拒绝。有什么帮助吗？

Answer 1

大声笑，我对解决方案感到as愧，在配置了群集之后，我没有重新启动，因此我的机器没有更新配置时给他们的新主机名。重新启动3台计算机后，我设法进入PySpark shell。

PySpark-连接被拒绝为主

1 个答案: