Question

如果我在独立应用程序中使用spark，我认为我不需要连接到服务器（当然）。但为什么我收到此网络错误消息？

[ERROR] Error while sending or receiving. 
Traceback (most recent call last): 
  File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/py4j/java_gateway.py", 
   line 473, in send_command 
    answer = smart_decode(self.stream.readline()[:-1]) 
  File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/socket.py", line 430, in readline 
    data = recv(1) 
timeout: timed out

....

Py4JNetworkError：尝试连接Java服务器时发生错误

我的代码如下：

from pyspark import * 
import pyspark.mllib.recommendation as spark_rec 
filename = "./yahoo music/train_0.txt" 
sc = SparkContext('local', 'spark_rec') 
aData = sc.textFile(filename).cache() (this part was successful, but the next following lines were failtures) 
ratings = aData.map(lambda line: np.array([float(x) for x in line.split('\t')])) 
rank = 10 
numIterations = 20 
aModel = spark_rec.ALS.train(ratings, rank, numIterations)

为什么独立应用程序在pyspark中使用套接字连接？

0 个答案: