Question

为什么以下代码失败？

import org.apache.spark.streaming._

val ssc = new StreamingContext(sc, Seconds(2))

val lStream = ssc.socketTextStream("127.0.0.1", 7777)

val words = lStream.flatMap(x => x.split(" ")).countByValue()

words.print()

ssc.start()

获得以下错误

17/07/27 17:34:08 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Restarting receiver with delay 2000ms:
Error connecting to 127.0.0.1:7777 - java.net.ConnectException: Connection refused                                              
        at java.net.PlainSocketImpl.socketConnect(Native Method)                                                               
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)                                        
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)                                 
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)                                          
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)                                                          
        at java.net.Socket.connect(Socket.java:579)                                                                            
        at java.net.Socket.connect(Socket.java:528)                                                                            
        at java.net.Socket.<init>(Socket.java:425)                                                                             
        at java.net.Socket.<init>(Socket.java:208)                                                                             
        at org.apache.spark.streaming.dstream.SocketReceiver.receive(SocketInputDStream.scala:73)                              
        at org.apache.spark.streaming.dstream.SocketReceiver$$anon$2.run(SocketInputDStream.scala:59)

Answer 1

错误消息是因为没有应用程序侦听7777上的127.0.0.1端口。

您可以使用nc实用程序来创建一个，但这只是为了演示目的。使用以下命令启动一个会话（为您使用的每个操作系统选择一个）。

// OpenBSD / macOS
nc -lk 7777

// other OSes
nc -lk -p 7777

请参阅Spark的文档中的A Quick Example。

为什么StreamingContext.socketTextStream给出“ERROR scheduler.ReceiverTracker：流0的注销接收器”？

1 个答案: