Question

我正在使用Scala 2.11.11，SBT 0.13.15和Kafka 0.10.2.1在Intelij IDEA 2017.2上制作Spark（2.2.0）流媒体项目。到目前为止，我已经能够导入其他流媒体模块，如org.apache.spark.streaming.StreamingContext和org.apache.spark.streaming.Seconds，但出于某种原因我的项目无法识别org.apache.spark.streaming.kafka010._，即使我相信我拥有合适的图书馆依赖关系：

name := "ProjectName"

version := "1.0"

resolvers += "Apache repo" at "https://repository.apache.org/content/repositories/releases"

val sparkVersion = "2.2.0"

libraryDependencies ++= Seq(
  "org.apache.spark" % "spark-core_2.11" % sparkVersion % "provided",
  "org.apache.spark" % "spark-streaming_2.11" % sparkVersion,
  "org.apache.kafka" % "kafka_2.11" % "0.10.2.1",
  "org.apache.logging.log4j" % "log4j-to-slf4j" % "2.8.2",
  "org.apache.logging.log4j" % "log4j" % "2.8.2" pomOnly(),
  "org.scalatest" %% "scalatest" % "3.0.1" % "test"
)

无论如何我可以解决这个问题吗？

Answer 1

此：

"org.apache.kafka" % "kafka_2.11" % "0.10.2.1"

不是正确的依赖。你需要：

"org.apache.spark" % "spark-streaming-kafka-0-10_2.11" % sparkVersion

无法导入org.apache.spark.streaming.kafka010

1 个答案: