结构化流式传输:无法从时间戳3中提取值:需要结构类型但有时间戳;

时间:2019-01-22 09:19:21

标签: apache-spark spark-structured-streaming

运行StructuredNetworkWordCountWindowed示例时出现错误。

Error: org.apache.spark.sql.AnalysisException: Can't extract value from timestamp#3: 
need struct type but got timestamp;

我已经用Google搜索,但是没有解决方案。这真是令人困惑。这是代码:

val words = lines.as[(String, Timestamp)].flatMap(line =>
      line._1.split(" ").map(word => (word, line._2))
    ).toDF("word", "timestamp")

val windowedCounts = words.groupBy(
  window($"timestamp", windowDuration, slideDuration)
).count().orderBy("window")

val query = windowedCounts.writeStream
  .outputMode("complete")
  .format("console")
  .option("truncate", "false")
  .start()

0 个答案:

没有答案