我是apache的新手一个尝试处理大查询的问题,当我尝试运行我在下面分享的代码时它给了我错误
文件“/****/***/spark-1.1.0/examples/src/main/python/testSql.py”,第25行,在 schemaEvents = sqlContext.createDataFrame(events) AttributeError:SQLContext实例没有属性'createDataFrame'
我正在关注练习的后续链接。 https://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes
代码在下面分享。
from pyspark.sql import *
from pyspark import *
#File = "/home/shafee/spark-1.1.0/results-20160315-075804.csv" # Should be some file on your system
File = "" # Should be some file on your system
sc = SparkContext()
sqlContext = SQLContext(sc)
Line = sc.textFile("results-20160315-075804.csv")
parts = Line.map(lambda l: l.split(","))
events = parts.map(lambda p: Row(Actor1Name=p[6], SourceUrl=int(p[56])))
schemaEvents = sqlContext.createDataFrame(events)
schemaPeople.registerTempTable("events")
df = sqlContext.sql("SELECT * FROM ").collect()
worldcount = df.count()
print("Totel words counts in csv file: %i" % (worldcount))