我正在尝试创建一个sbt Scala项目,从Flink添加一些代码。我已经成功添加了大部分Flink项目,但是Flink-ML。
我尝试添加不同的依赖项,错误如下:
sbt.build配置:
resolvers += Resolver.mavenLocal
// I have downloaded the Flink project and compiled in local.
libraryDependencies += "org.apache.flink" % "flink-ml_2.10" % "1.0-SNAPSHOT"
错误如下:
Error:Error while importing SBT project:<br/>...<br/><pre>[info] Resolving org.scala-sbt#cache;0.13.8 ...
[info] Resolving org.scala-sbt#testing;0.13.8 ...
[info] Resolving org.scala-sbt#test-agent;0.13.8 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.8 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.8 ...
[info] Resolving org.scala-sbt#command;0.13.8 ...
[info] Resolving org.scala-sbt#logic;0.13.8 ...
[info] Resolving org.scala-sbt#precompiled-2_8_2;0.13.8 ...
[info] Resolving org.scala-sbt#precompiled-2_9_2;0.13.8 ...
[info] Resolving org.scala-sbt#precompiled-2_9_3;0.13.8 ...
[error] Modules were resolved with conflicting cross-version suffixes in {file:/home/ubuntu/workspace/Flink-project-sbt/}flink-project-sbt:
[error] org.apache.flink:flink-runtime <none>, _2.10
[error] org.apache.flink:flink-scala <none>, _2.10
[error] org.apache.flink:flink-optimizer <none>, _2.10
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) Conflicting cross-version suffixes in: org.apache.flink:flink-runtime, org.apache.flink:flink-scala, org.apache.flink:flink-optimizer
[error] (*:ssExtractDependencies) Conflicting cross-version suffixes in: org.apache.flink:flink-runtime, org.apache.flink:flink-scala, org.apache.flink:flink-optimizer
[error] Total time: 33 s, completed Feb 23, 2016 8:56:38 AM</pre><br/>See complete log in <a href="/home/ubuntu/.IntelliJIdea15/system/log/sbt.last.log">/home/ubuntu/.IntelliJIdea15/system/log/sbt.last.log</a>
我也尝试用flink ml for 2.11
libraryDependencies += "org.apache.flink" % "flink-ml_2.11" % "0.10.1"
错误如下:
Error:Error while importing SBT project:<br/>...<br/><pre>[error] org.apache.flink:flink-scala <none>, _2.11
[error] org.apache.flink:flink-optimizer <none>, _2.11
[error] org.apache.flink:flink-shaded-hadoop2 <none>, _2.11
[error] org.clapper:grizzled-slf4j _2.10, _2.11
[error] com.typesafe.akka:akka-actor _2.10, _2.11
[error] com.twitter:chill _2.10, _2.11
[error] com.twitter:bijection-core _2.10, _2.11
[error] com.twitter:bijection-avro _2.10, _2.11
[error] org.apache.flink:flink-core <none>, _2.11
[error] com.twitter:chill-bijection _2.10, _2.11
[error] com.twitter:chill-avro _2.10, _2.11
[error] com.github.scopt:scopt _2.10, _2.11
[error] com.typesafe.akka:akka-remote _2.10, _2.11
[error] org.apache.flink:flink-java <none>, _2.11
[error] com.typesafe.akka:akka-slf4j _2.10, _2.11
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) Conflicting cross-version suffixes in: org.apache.flink:flink-runtime, org.apache.flink:flink-scala, org.apache.flink:flink-optimizer, org.apache.flink:flink-shaded-hadoop2, org.clapper:grizzled-slf4j, com.typesafe.akka:akka-actor, com.twitter:chill, com.twitter:bijection-core, com.twitter:bijection-avro, org.apache.flink:flink-core, com.twitter:chill-bijection, com.twitter:chill-avro, com.github.scopt:scopt, com.typesafe.akka:akka-remote, org.apache.flink:flink-java, com.typesafe.akka:akka-slf4j
[error] (*:ssExtractDependencies) Conflicting cross-version suffixes in: org.apache.flink:flink-runtime, org.apache.flink:flink-scala, org.apache.flink:flink-optimizer, org.apache.flink:flink-shaded-hadoop2, org.clapper:grizzled-slf4j, com.typesafe.akka:akka-actor, com.twitter:chill, com.twitter:bijection-core, com.twitter:bijection-avro, org.apache.flink:flink-core, com.twitter:chill-bijection, com.twitter:chill-avro, com.github.scopt:scopt, com.typesafe.akka:akka-remote, org.apache.flink:flink-java, com.typesafe.akka:akka-slf4j
[error] Total time: 21 s, completed Feb 23, 2016 8:59:12 AM</pre><br/>See complete log in <a href="/home/ubuntu/.IntelliJIdea15/system/log/sbt.last.log">/home/ubuntu/.IntelliJIdea15/system/log/sbt.last.log</a>
我的sbt文件内容是:
name := "Flink-project-sbt"
version := "1.0"
scalaVersion := "2.10.6"
libraryDependencies ++= Seq("org.apache.flink" % "flink-scala" % "0.10.2")
libraryDependencies ++= Seq("org.apache.flink" % "flink-streaming-scala" % "0.10.2", "org.apache.flink" % "flink-clients" % "0.10.2")
// Table API dependencies
libraryDependencies += "org.apache.flink" % "flink-table" % "0.10.1"
// Machine Learning
libraryDependencies += "org.apache.flink" % "flink-ml_2.11" % "0.10.1"
// Machine Learning
resolvers += Resolver.mavenLocal
//libraryDependencies += "org.apache.flink" % "flink-ml_2.10" % "1.0-SNAPSHOT"
// Gelly
libraryDependencies += "org.apache.flink" % "flink-gelly-scala" % "0.10.1"
// for more dependencies see below url
// http://mvnrepository.com/artifact/org.apache.flink
fork in run := true
flink ml中是否有与scala 2.10兼容的版本?
谢谢!
答案 0 :(得分:2)
问题在于您正在混合版本0.10.2
和版本1.0-SNAPSHOT
。在后一版本中,我们已正确引入了Scala版本后缀_2.10
和_2.11
。此外,我们已经纠正了一些依赖性问题,这些问题阻止了sbt程序集插件构建胖罐。因此,我强烈建议使用版本为1.0-SNAPSHOT
的当前版本。
可能的sbt文件可能看起来像
resolvers in ThisBuild ++= Seq(Resolver.mavenLocal)
name := "flinkMLTest"
version := "0.1-SNAPSHOT"
organization := "org.example"
scalaVersion in ThisBuild := "2.10.6"
val flinkVersion = "1.0-SNAPSHOT"
val flinkDependencies = Seq(
"org.apache.flink" %% "flink-scala" % flinkVersion % "provided",
"org.apache.flink" %% "flink-ml" % flinkVersion)
lazy val root = (project in file(".")).
settings(
libraryDependencies ++= flinkDependencies
)
// make run command include the provided dependencies
run in Compile <<= Defaults.runTask(fullClasspath in Compile, mainClass in (Compile, run), runner in (Compile, run))
还有一个用于Flink作业的新giter8模板。如果您有giter8 installed,那么您只需发出以下命令即可使用Scala和SBT生成Flink项目。
g8 tillrohrmann/flink-project
答案 1 :(得分:0)
_2.10
或_2.11
)flink-scala
变为flink-scala_2.10
和flink-scala_2.11
)。这些后缀已在当前的主人中使用因此,您需要决定使用Flink的版本(以及当前版本的Scala版本),并以合并的方式指定依赖项。