Question

可以配置Spark，以便不是绑定到端口7077的地址127.0.1.1，而是可以而是绑定到0.0.0.0。与端口8080绑定的方式相同：

netstat -pln
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.1.1:7077          0.0.0.0:*               LISTEN      2864/java
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      2864/java
tcp        0      0 127.0.1.1:6066          0.0.0.0:*               LISTEN      2864/java
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -
udp        0      0 192.168.192.22:123      0.0.0.0:*                           -
udp        0      0 127.0.0.1:123           0.0.0.0:*                           -
udp        0      0 0.0.0.0:123             0.0.0.0:*                           -
udp        0      0 0.0.0.0:21415           0.0.0.0:*                           -
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Path
unix  2      [ ACC ]     STREAM     LISTENING     7195     -                   /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     SEQPACKET  LISTENING     405      -                   /run/udev/control

原因我问这是因为我无法将工作人员连接到主节点，我认为问题在于主IP无法被发现。

尝试将slave连接到master时出错：

15/04/02 21:58:18 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@raspberrypi:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: Connection refused: raspberrypi/192.168.192.22:7077
15/04/02 21:58:18 INFO RemoteActorRefProvider$RemoteDeadLetterActorRef: Message [org.apache.spark.deploy.DeployMessages$RegisterWorker] from Actor[akka://sparkWorker/user/Worker#1677101765] to Actor[akka://sparkWorker/deadLetters] was not delivered. [10] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

Answer 1

在spark-env.sh中，您可以设置SPARK_MASTER_IP=<ip>。

主机名也可以正常工作（通过SPARK_STANDALONE_MASTER=<hostname>），只需确保工作人员连接到与主人绑定的主机名完全相同的主机名（即Spark主UI中显示的spark：//地址）

更改端口7077上运行的绑定IP - Apache Spark

1 个答案: