Question

我无法从终端启动pyspark shell。 Python包：Anaconda 4.3.0（Python 3.6） Spark：spark-2.1.0

以下是我收到的错误的屏幕截图：

rahul@rahul-GA-78LMT-USB3-6-0:~/Hadoop/spark$ pyspark
   Python 3.6.0 |Anaconda 4.3.0 (64-bit)| (default, Dec 23 2016, 12:22:00) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/home/rahul/Hadoop/spark/python/pyspark/shell.py", line 30, in <module>
    import pyspark
  File "/home/rahul/Hadoop/spark/python/pyspark/__init__.py", line 44, in <module>
    from pyspark.context import SparkContext
  File "/home/rahul/Hadoop/spark/python/pyspark/context.py", line 36, in <module>
    from pyspark.java_gateway import launch_gateway
  File "/home/rahul/Hadoop/spark/python/pyspark/java_gateway.py", line 31, in <module>
    from py4j.java_gateway import java_import, JavaGateway, GatewayClient
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/home/rahul/Hadoop/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 18, in <module>
  File "/home/rahul/anaconda3/lib/python3.6/pydoc.py", line 62, in <module>
    import pkgutil
  File "/home/rahul/anaconda3/lib/python3.6/pkgutil.py", line 22, in <module>
    ModuleInfo = namedtuple('ModuleInfo', 'module_finder name ispkg')
  File "/home/rahul/Hadoop/spark/python/pyspark/serializers.py", line 393, in namedtuple
    cls = _old_namedtuple(*args, **kwargs)
TypeError: namedtuple() missing 3 required keyword-only arguments: 'verbose', 'rename', and 'module'

以下是我的.bashrc：

export JAVA_HOME=/home/rahul/Hadoop/java
export HADOOP_HOME=/home/rahul/Hadoop/hadoop-2.7.3
export HIVE_HOME=/home/rahul/MSME/tarballs/hive
export SCALA_HOME=/home/rahul/Hadoop/scala
export SPARK_HOME=/home/rahul/Hadoop/spark
export ANACONDA_HOME=/home/rahul/anaconda3
export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH
export PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME/bin:$SCALA_HOME/bin:$SPARK_HOME/bin:$ANACONDA_HOME/bin:$PATH

无法启动pyspark shell

0 个答案: