Pyspark LDA:TypeError:'JavaPackage'对象不可调用

时间:2018-02-28 16:47:15

标签: apache-spark pyspark apache-spark-mllib pyspark-sql lda

我在尝试初始化spark上下文时安装了pyspark并出现类型错误。

Pyspark安装: enter image description here

代码如下:

 import pyspark
from collections import defaultdict
from pyspark import SparkContext
from pyspark.mllib.linalg import Vector, Vectors
from pyspark.mllib.clustering import LDA, LDAModel
from pyspark.sql import SQLContext

# Initialize
sc = SparkContext("local", "Simple App")
sql_context = SQLContext(sc)

错误信息如下:

    ---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-bedceba80d96> in <module>()
      7 
      8 # Initialize
----> 9 sc = SparkContext("local", "Simple App")
     10 sql_context = SQLContext(sc)

c:\users\vishn\appdata\local\enthought\canopy\user\lib\site-packages\pyspark\context.pyc in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
    116         try:
    117             self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,
--> 118                           conf, jsc, profiler_cls)
    119         except:
    120             # If an error occurs, clean up in order to allow future SparkContext creation:

c:\users\vishn\appdata\local\enthought\canopy\user\lib\site-packages\pyspark\context.pyc in _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, jsc, profiler_cls)
    186         self._accumulatorServer = accumulators._start_update_server()
    187         (host, port) = self._accumulatorServer.server_address
--> 188         self._javaAccumulator = self._jvm.PythonAccumulatorV2(host, port)
    189         self._jsc.sc().register(self._javaAccumulator)
    190 

TypeError: 'JavaPackage' object is not callable

0 个答案:

没有答案