Question

我正在尝试运行Streaming SQL（https://github.com/intel-spark/stream-sql）示例wordcount。这反过来又使用催化剂项目。我能够跑 sbt包没有任何问题。但是，当我确实引发了scala代码的提交时，我遇到了以下问题。

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.trees.UnaryNode$class.children(Lorg/apache/spark/sql/catalyst/trees/UnaryNode;)Lscala/collection/immutable/List;
    at org.apache.spark.sql.streaming.WindowedPhysicalPlan.children(windowedPlans.scala:39)
    at org.apache.spark.sql.streaming.WindowedPhysicalPlan.children(windowedPlans.scala:39)
    at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$5.apply(TreeNode.scala:264)
    at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)

我已正确包含了dependancies和classfile。以下是.sbt文件的外观

name := "Word Count"

version := "1.0"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.3.1"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.3.1"
libraryDependencies += "org.apache.spark" %% "spark-catalyst" % "1.3.1"
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.3.1"
libraryDependencies += "spark.streamsql" %% "spark-streamsql" % "0.1.0-SNAPSHOT"

Answer 1

这只是一个版本问题，删除所有火花罐并从apache spark lib文件夹中添加spark-assembly-1.3.0-hadoop2.4.0.jar。

如何从Streaming SQL运行WordCount示例（抛出NoSuchMethodError）？

1 个答案: