无法在Windows

时间:2017-04-04 07:24:34

标签: apache-spark pyspark

一些背景知识: 1.运行Windows 7 2.我安装了python 2.7 3.我重新安装了JDK 8 4.我安装了Spark并按照步骤设置环境变量等(参考 - https://nerdsrule.co/2016/06/15/ipython-notebook-and-spark-setup-for-windows-10/) 当我运行spark-shell时,它会引发大量错误,但最终会向我展示" SPARK"标志和斯卡拉终端。

问题: -

但是当我运行pyspark时,我收到以下错误: enter image description here



PS C:\Users\harleen.singh> pyspark
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
17/04/04 15:18:47 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java cl
asses where applicable
Traceback (most recent call last):
  File "C:\Spark\bin\..\python\pyspark\shell.py", line 43, in <module>
    spark = SparkSession.builder\
  File "C:\Spark\python\pyspark\sql\session.py", line 179, in getOrCreate
    session._jsparkSession.sessionState().conf().setConfString(key, value)
  File "C:\Spark\python\lib\py4j-0.10.4-src.zip\py4j\java_gateway.py", line 1133, in __call__
  File "C:\Spark\python\pyspark\sql\utils.py", line 79, in deco
    raise IllegalArgumentException(s.split(': ', 1)[1], stackTrace)
pyspark.sql.utils.IllegalArgumentException: u"Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':"
&#13;
&#13;
&#13;

在我收到的python提示符中,我得到以下输出: enter image description here

&#13;
&#13;
>>> sc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'sc' is not defined
>>> SparkContext
<class 'pyspark.context.SparkContext'>
>>>
&#13;
&#13;
&#13;

我需要做什么?

0 个答案:

没有答案