java.lang.NoSuchMethodError：akka.actor.ActorCell.addFunctionRef

时间：2016-02-23 13:22:34

标签： scala sbt akka sbt-assembly akka-http

我正在尝试设置一个简单的akka​​-http 2.4.2项目来测试它，但我没有这样做。

我的built.sbt：

import NativePackagerHelper._

lazy val akkaVersion = "2.4.2"

lazy val root = (project in file(".")).
settings(
name := "akkTest",
version := "0.1",
scalaVersion := "2.11.7")

libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-http-spray-json-experimental" % akkaVersion
)

enablePlugins(JavaServerAppPackaging)

我在Main.scala中的代码片段

import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.actor.ActorSystem


object Main extends App {

implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
implicit val ec = system.dispatcher

val serverSource =
    Http().bind(interface = "localhost", port = 8080)
val bindingFuture =
    serverSource.to(Sink.foreach { connection => // foreach materializes the source
    println("Accepted new connection from " + connection.remoteAddress)
    }).run()

}

执行错误抛出：

Uncaught error from thread [default-akka.actor.default-dispatcher-2] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[default]
java.lang.NoSuchMethodError: akka.actor.ActorCell.addFunctionRef(Lscala/Function2;)Lakka/actor/FunctionRef;
        at akka.stream.stage.GraphStageLogic$StageActor.<init>(GraphStage.scala:143)
        at akka.stream.stage.GraphStageLogic.getStageActor(GraphStage.scala:904)
        at akka.stream.impl.io.ConnectionSourceStage$$anon$1.preStart(TcpStages.scala:56)
        at akka.stream.impl.fusing.GraphInterpreter.init(GraphInterpreter.scala:468)
        at akka.stream.impl.fusing.GraphInterpreterShell.init(ActorGraphInterpreter.scala:363)
        at akka.stream.impl.fusing.ActorGraphInterpreter.tryInit(ActorGraphInterpreter.scala:502)
        at akka.stream.impl.fusing.ActorGraphInterpreter.preStart(ActorGraphInterpreter.scala:539)
        at akka.actor.Actor$class.aroundPreStart(Actor.scala:472)
        at akka.stream.impl.fusing.ActorGraphInterpreter.aroundPreStart(ActorGraphInterpreter.scala:493)
        at akka.actor.ActorCell.create(ActorCell.scala:580)
        at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
        at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
        at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:279)
        at akka.dispatch.Mailbox.run(Mailbox.scala:220)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

这必须是我的环境，但我不知道如何跟踪问题。我使用的是jdk 1.8u71 

[info] Done updating.
[info] Including from cache: ssl-config-akka_2.11-0.1.3.jar
[info] Including from cache: reactive-streams-1.0.0.jar
[info] Including from cache: akka-http-spray-json-experimental_2.11-2.4.2.jar
[info] Including from cache: config-1.3.0.jar
[info] Including from cache: spray-json_2.11-1.3.2.jar
[info] Including from cache: ssl-config-core_2.11-0.1.3.jar
[info] Including from cache: scala-parser-combinators_2.11-1.0.4.jar
[info] Including from cache: scala-java8-compat_2.11-0.7.0.jar
[info] Including from cache: akka-parsing_2.11-2.4.2.jar
[info] Including from cache: akka-http-experimental_2.11-2.4.2.jar
[info] Including from cache: akka-actor_2.11-2.4.2.jar
[info] Including from cache: akka-http-core_2.11-2.4.2.jar
[info] Including from cache: akka-stream_2.11-2.4.2.jar
[info] Including from cache: scala-library-2.11.7.jar

请记住，我只指出相同akka版本的依赖关系

此程序在使用 sbt run 时工作正常但在使用我自己的scala启动器使用已组装的jar时失败

5 个答案:

答案 0 :(得分：5)

问题是Spark内部使用Akka。只要您将Spark Job作为自包含的应用程序（例如sbt run）运行，这不是问题，因为将使用您自己的Akka版本。但是，只要您将应用程序提交到具有spark-submit的群集，情况就会发生变化。然后，Spark类加载器将通过捆绑在sparkJob.jar中的Akka实现选择Akka的内部版本。因此，来自上方的NoSuchMethodError来自akka-stream_2.11-2.4.2调用akka-actor_2.11-2.3.x.jar，这是在Spark中使用，而不是akka-actor_2.11-2.4.2.jar在您的工作中捆绑在一起。方法addFunctionRef实际上是一个非常recent addition，并且在早期版本的Akka中不存在。您可以通过在发生异常的位置设置断点（或使用异常断点）来验证这一点。当应用程序暂停在GraphStage中有问题的位置时，请评估

materializer.supervsor.getClass().getResource("ActorCell.class")

这将打印出加载的类文件ActorCell的位置。

为确保您与Spark内部使用的Akka版本隔离，您可以使用--driver-class-path的{​​{1}}选项，例如

spark-submit

如果您这样做，我还建议您在spark-submit --class MyJob \ --driver-class-path akka-actor_2.11-2.4.2.jar \ --master spark://master:7077 \ sparkJob.jar 中将akka-actor设置为"test, provided"，这样您就不会在{{1}中加入build.sbt }}

答案 1 :(得分：1)

当这件事发生在我身上时，我通过消除对旧的akka​​-actor版本的传递依赖来解决它。尽管依赖关系看起来很明显，但我建议您仔细查看依赖树，看看是否包含了早期版本的akka​​-actor。

答案 2 :(得分：1)

Scala 2.11捆绑了旧版本的akka​​-actor。 Scala使他们的库优先于用户创建的库。 Spark可能会捆绑相同的，虽然在它的装配Jar内。

在scala中，从lib文件夹中删除akka jar就足够了。在Spark中，利用驱动程序类路径解决问题，或利用SQLContext配置使用户库优先

未回答的问题是为什么在所有设置中都不会发生这种情况，因为这在家里没有发生（也许OS存储库不会使用akka-actor库安装scala）

https://github.com/sbt/sbt-assembly/issues/204

答案 3 :(得分：1)

对于那些即使使用sbt run也会发生的情况。我的解决方案是在akka-stream文件中添加build.sbt依赖项，例如：

  "com.typesafe.akka" %% "akka-stream" % "2.5.3",
  "com.typesafe.akka" %% "akka-stream-testkit" % "2.5.3" % Test

答案 4 :(得分：0)

将akka-http-experimental-2.4.11.2与akka-actor-2.5.6一起使用时，我遇到了类似的问题。

dependencies {
    compile group: 'org.scala-lang', name: 'scala-library', version: '2.11.11'
    compile group: 'com.typesafe.akka', name: 'akka-actor_2.11', version: '2.5.6'

    compile group: 'com.typesafe.akka', name: 'akka-http-experimental_2.11', version: '2.4.11.2'
}

问题是，akka-http-experimental-2.4.11.2本身使用的com.typesafe.akka:akka-actor:2.4.11.2。被2.5.6覆盖。

./gradlew dependencies

compileClasspath - Compile classpath for source set 'main'.
+--- org.scala-lang:scala-library:2.11.11
+--- com.typesafe.akka:akka-actor_2.11:2.5.6
|    +--- org.scala-lang:scala-library:2.11.11
|    +--- com.typesafe:config:1.3.1
|    \--- org.scala-lang.modules:scala-java8-compat_2.11:0.7.0
|         \--- org.scala-lang:scala-library:2.11.7 -> 2.11.11
+--- com.typesafe.akka:akka-http-experimental_2.11:2.4.11.2
|    +--- org.scala-lang:scala-library:2.11.8 -> 2.11.11
|    \--- com.typesafe.akka:akka-http-core_2.11:2.4.11.2
|         +--- org.scala-lang:scala-library:2.11.8 -> 2.11.11
|         +--- com.typesafe.akka:akka-stream_2.11:2.4.11.2
|         |    +--- org.scala-lang:scala-library:2.11.8 -> 2.11.11
|         |    +--- com.typesafe.akka:akka-actor_2.11:2.4.11.2 -> 2.5.6 (*)  <---- Ting
|         |    +--- com.typesafe:ssl-config-akka_2.11:0.2.1
|         |    |    +--- org.scala-lang:scala-library:2.11.8 -> 2.11.11
|         |    |    \--- com.typesafe:ssl-config-core_2.11:0.2.1
|         |    |         +--- org.scala-lang:scala-library:2.11.8 -> 2.11.11
|         |    |         +--- com.typesafe:config:1.2.0 -> 1.3.1
|         |    |         \--- org.scala-lang.modules:scala-parser-combinators_2.11:1.0.4
|         |    |              \--- org.scala-lang:scala-library:2.11.6 -> 2.11.11
|         |    \--- org.reactivestreams:reactive-streams:1.0.0
|         \--- com.typesafe.akka:akka-parsing_2.11:2.4.11.2
|              \--- org.scala-lang:scala-library:2.11.8 -> 2.11.11
\--- org.codehaus.groovy:groovy-all:2.4.12

(*) - dependencies omitted (listed previously)

1）我将akka-actor版本降低到2.4.11后，工作正常。 

dependencies {
    compile group: 'org.scala-lang', name: 'scala-library', version: '2.11.11'
    compile group: 'com.typesafe.akka', name: 'akka-actor_2.11', version: '2.4.11'
    compile group: 'com.typesafe.akka', name: 'akka-http-experimental_2.11', version: '2.4.11.2'
}

2）另一种方法是exclude不受欢迎的akka_actor并明确添加想要的内容。 

dependencies {
    compile group: 'org.scala-lang', name: 'scala-library', version: '2.11.11'
    compile group: 'com.typesafe.akka', name: 'akka-actor_2.11', version: '2.5.6'
    compile group: 'com.typesafe.akka', name: 'akka-http-experimental_2.11', version: '2.4.11.2' {
       exclude group: "com.typesafe.akka", module: "akka-actor_2.11"         // <- exlude actor 2.4.11.2
    }
}
相关问题
最新问题