尝试在Spring框架中启动Spark应用程序时出现以下错误

时间:2018-08-07 06:38:57

标签: apache-spark

  

线程“主”中的异常java.lang.NoSuchMethodError:> org.apache.spark.internal.config.TypedConfigBuilder.checkValue(Lscala / Function1;> Ljava / lang / String;)Lorg / apache / spark / internal / config / TypedConfigBuilder;     在org.apache.spark.sql.internal.SQLConf $。(SQLConf.scala:276)     在org.apache.spark.sql.internal.SQLConf $。(SQLConf.scala)     在org.apache.spark.sql.internal.StaticSQLConf $。>(StaticSQLConf.scala:31)     在org.apache.spark.sql.internal.StaticSQLConf $。(StaticSQLConf.scala)     在> org.apache.spark.sql.SparkSession $ Builder.getOrCreate(SparkSession.scala:935)     在com.example.demo.SparkDemo.main(SparkDemo.java:19)

这是我的Java代码:

`public static void main(String [] args){

    SparkConf conf= new SparkConf().setAppName("Java Spark").setMaster("local[*]");

    SparkSession spark = SparkSession
            .builder()
            .config(conf)
            .getOrCreate();

    JavaSparkContext sc = new JavaSparkContext();
    JavaRDD<String> lines=sc.textFile("E://Car.txt");
    JavaRDD<String> filteredLines = lines.filter(new Function<String, Boolean>() {

        @Override
        public Boolean call(String arg0) throws Exception {
            // TODO Auto-generated method stub
            return true;
        }
    });

}

` 这是我的spark的pom.xml依赖项:

  

               org.apache.spark             spark-core_2.11             2.0.0                                  org.apache.spark             spark-sql_2.11             2.3.0         

0 个答案:

没有答案