AttributeError:SQLContext实例没有属性'createDataFrame'

时间:2016-03-15 21:22:09

标签: python apache-spark pyspark apache-spark-sql

我是apache的新手一个尝试处理大查询的问题,当我尝试运行我在下面分享的代码时它给了我错误

  

文件“/****/***/spark-1.1.0/examples/src/main/python/testSql.py”,第25行,在       schemaEvents = sqlContext.createDataFrame(events)   AttributeError:SQLContext实例没有属性'createDataFrame'

我正在关注练习的后续链接。 https://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes

代码在下面分享。

from pyspark.sql import *


from pyspark import *
#File = "/home/shafee/spark-1.1.0/results-20160315-075804.csv"  # Should be some file on your system
File = ""  # Should be some file on your system
sc = SparkContext()
sqlContext = SQLContext(sc)
Line = sc.textFile("results-20160315-075804.csv")
parts = Line.map(lambda l: l.split(","))
events = parts.map(lambda p: Row(Actor1Name=p[6], SourceUrl=int(p[56])))

schemaEvents = sqlContext.createDataFrame(events)

schemaPeople.registerTempTable("events")
df = sqlContext.sql("SELECT * FROM ").collect()
worldcount = df.count()
print("Totel words counts in csv file: %i" % (worldcount))

1 个答案:

答案 0 :(得分:0)

您使用Spark 1.1并在Spark 1.3中引入了SQLContext.createDataFrame。如果要使用当前的编程指南,则应更新Spark版本。