我正在使用Spark Streaming Twitter包来获取流行的主题标签。具体来说,我正在尝试复制此代码https://github.com/andrewrgoss/udemy-spark-scala/tree/master/twitter_streaming
我在Windows命令行上运行了以下命令:
sbt package
spark-submit --packages org.apache.bahir:spark-streaming-twitter_2.11:2.1.0 --class com.andrewrgoss.spark.PopularHashtags \ target \ scala-2.11 \ twitter_streaming_2.11-0.1.jar
这给我以下错误。
2018-04-20 03:16:30 INFO BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, 172.24.20.2, 56082, None)
2018-04-20 03:16:30 INFO BlockManager:54 - Initialized BlockManager: BlockManagerId(driver, 172.24.20.2, 56082, None)
2018-04-20 03:16:30 INFO ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6b00ad9{/metrics/json,null,AVAILABLE,@Spark}
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:106)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
-------------------------------------------
Time: 1524208600000 ms
-------------------------------------------
-------------------------------------------
Time: 1524208610000 ms
-------------------------------------------
-------------------------------------------
我找到了相关的帖子Spark Program finding Popular HashTags from twiiter,但没有解决方法。来自社区的人是否可以详细说明错误以及如何解决错误?