线程“main”中的异常java.lang.NoClassDefFoundError：org / apache / spark / streaming / StreamingContext

Question

Hie everyone，看起来在以下代码中找不到类StreamingContext。

import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.{SparkConf, SparkContext}
object Exemple {
  def main(args: Array[String]): Unit = {
    val conf = new SparkConf().setMaster("local[*]").setAppName("Exemple")
    val sc = new SparkContext(conf)
    val ssc = new StreamingContext(sc, Seconds(2)) //this line throws error

  }
}

这是错误：

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/streaming/StreamingContext
    at Exemple$.main(Exemple.scala:16)
    at Exemple.main(Exemple.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.StreamingContext
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 2 more

Process finished with exit code 1

我使用以下build.sbt文件：

name := "exemple"

version := "1.0.0"

scalaVersion := "2.11.11"

// https://mvnrepository.com/artifact/org.apache.spark/spark-sql
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.2.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-streaming
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "2.2.0" % "provided"
// https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10
libraryDependencies += "org.apache.spark" %% "spark-streaming-kafka-0-10" % "2.2.0"

我使用intellij Run按钮运行Exemple类，我收到错误。在sbt shell中它工作正常。进入我的dependecies'module，我可以找到火花依赖。代码在intellij中编译。我可以在外部库中看到spark依赖（在左侧项目面板内）。 你有什么主意吗。这似乎并不复杂。

Answer 1

请删除spark-streaming库提供的内容。

self.aRent

更改后，仍然存在依赖性问题，排除重复的jar。

Potential Input: 1000, 1, 1
Desired Output: 
Aparment#  Rent   Rooms  Bathrooms
111        1000   1      `1
class Apartments:
     def __init__(self, apartNum, rent, rooms, bath):
        self.apartnum=apartNum
        self.rent=rent
        self.rooms=rooms
        self.baths=bath

     def setRent(self,rent):
        self.rent = rent
     def getRent(self):
        return self.rent

     def setRooms(self,rooms):
        self.rooms = rooms
     def getRooms(self):
        return self.rooms

     def setBaths(self,baths):
        self.baths = baths
     def getBaths(self):
        return self.baths

apt_1= Apartments(111,1000,1,1)
apt_2= Apartments(112,2000,2,1)
apt_3= Apartments(113,2500,2,2)
apt_4= Apartments(114,3000,3,2)
apt_5= Apartments(115,3500,3,3)

rentInput=input('Enter maximum amount of rent: ')    
roomInput=input('Enter minimum number of bedrooms: ')
bathInput=input('Enter minimum number of baths: ')

希望这有帮助。

由于 拉维