Question

我正在尝试使用Spyder（IDE）中的pyspark初始化spark，但始终会出错。（操作系统-Ubuntu 16.04 LTS）

from pyspark.sql import SparkSession
spark = SparkSession.builder.getOrCreate()

File "<ipython-input-7-86b3963ebca1>", line 3, in <module>
    .appName("test_import")\

  File "/home/fly/anaconda3/lib/python3.7/site-packages/pyspark/sql/session.py", line 173, in getOrCreate
    sc = SparkContext.getOrCreate(sparkConf)

  File "/home/fly/anaconda3/lib/python3.7/site-packages/pyspark/context.py", line 349, in getOrCreate
    SparkContext(conf=conf or SparkConf())

  File "/home/fly/anaconda3/lib/python3.7/site-packages/pyspark/context.py", line 118, in __init__
    conf, jsc, profiler_cls)

  File "/home/fly/anaconda3/lib/python3.7/site-packages/pyspark/context.py", line 187, in _do_init
    self._accumulatorServer = accumulators._start_update_server(auth_token)

  File "/home/fly/anaconda3/lib/python3.7/site-packages/pyspark/accumulators.py", line 291, in _start_update_server
    server = AccumulatorServer(("localhost", 0), _UpdateRequestHandler, auth_token)

  File "/home/fly/anaconda3/lib/python3.7/site-packages/pyspark/accumulators.py", line 274, in __init__
    SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass)

  File "/home/fly/anaconda3/lib/python3.7/socketserver.py", line 452, in __init__
    self.server_bind()

  File "/home/fly/anaconda3/lib/python3.7/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)

gaierror: [Errno -2] Name or service not known

Answer 1

这很可能是因为您无法与spark-master连接。转到您的/etc/hosts文件，并在其中添加此行。

127.0.0.1       localhost

无法初始化Spark

1 个答案: