Question

我试图将现有列表的值添加到元组中。它没有显示任何编译器错误，但会抛出运行时错误。

我对一些列表值进行了硬编码，并尝试使用Java Spark将这些值添加到元组中。

    @SuppressWarnings("unchecked")
    public static void main(String[] args)  {

        List<Integer> arr=new ArrayList<Integer>();
        arr.add(1);
        arr.add(4);
        arr.add(16);
        arr.add(16);
        arr.add(25);
        arr.add(36);
        arr.add(49);
        arr.add(64);
        arr.add(100);



        Logger.getLogger("org.apache").setLevel(Level.WARN);
        System.setProperty("hadoop.home.dir", "C:\\packages_for_spark\\hadoop-2.6.0");

            SparkConf sparkConf = new SparkConf().setAppName("First_spark");
            sparkConf.set("spark.master", "local")
            .set("spark.driver.allowMultipleContexts", "true");
         SparkContext sc = new SparkContext(sparkConf);
           JavaSparkContext stc = new JavaSparkContext(sparkConf);




          JavaRDD<Integer> num = stc.parallelize(arr);



          JavaRDD<Tuple7<Integer, Integer, Integer, Integer, Integer, Integer, Integer>> mapped=
                  num.map((rd) -> {
                      List<Integer> rd1= num.collect();

                     return new Tuple7<>(rd1.get(0),rd1.get(1),rd1.get(2),rd1.get(3),rd1.get(4),rd1.get(5),rd1.get(6));



                  });


          mapped.collect().forEach(System.out::println);


    }}

它不会引发任何编译器错误，但在运行时会显示以下错误：

WARN SparkContext: Multiple running SparkContexts detected in the same JVM!
org.apache.spark.SparkException: Only one SparkContext may be running in this JVM (see SPARK-2243). To ignore this error, set spark.driver.allowMultipleContexts = true. The currently running SparkContext was created at:
org.apache.spark.SparkContext.<init>(SparkContext.scala:76)

Answer 1

您可以尝试更改为

JavaSparkContext jsc = JavaSparkContext.fromSparkContext(sc);

如何将列表值添加到元组

1 个答案: