sparksession中的错误,因为无法解决sparksession

时间:2019-01-21 08:41:40

标签: java apache-spark

我是引发相关工作的新手。我尝试了如下代码。 包hdd.models;

<!doctype html>
<html lang="en">

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
</head>

<body>

  <?php 
    
    $phpInfo = array();
    
    $links = array(
     array('example.html', 'ex', 'Example'),
     array('example.html', 'ex', 'Example-2'),
    );
    
    // add $links to $phpInfo
    
    if( !isset($phpInfo['links']) ){
      $phpInfo['links'] = array();
    }
    
    $phpInfo['links'] = $links; // now we have $links in our "global" variable
  
  ?>

  <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>


  <script id="hold-php-data" data-php='<?php echo json_encode($phpInfo); ?>'></script>

  <script>
    $(function() {

      var mockupElement = $('#hold-php-data');
      var phpInfo = JSON.parse(mockupElement.attr('data-php') || []); // get the json data from attribute and convert it to a valid object

      // we can safely remove the mockupElement since we have the data so it won't be exposed to the user later on.
      mockupElement.remove();

      // now you can access the links as phpInfo.links

      console.log('data from php', phpInfo);

    });
  </script>

</body>

</html>

sparksession中发生以下错误 “无法解析org.apache.spark.sql.SparkSession $ Builder类型。它是从所需的.class文件中间接引用的” 注意:我使用的是带有scala 2.10的spark-2.1.0。上面的代码我在java eclipse-neon中尝试过

2 个答案:

答案 0 :(得分:0)

没有意义使用构建器。 只需在开始时创建Spark Session,然后从session调用spark上下文。

SparkSession sparkSession = SparkSession.builder().config("spark.serializer", "org.apache.spark.serializer.KryoSerializer").config("spark.kryo.registrator", "org.datasyslab.geospark.serde.GeoSparkKryoRegistrator").master("local[*]").appName("testGeoSpark").getOrCreate();
sparkSession.sparkContext().textFile(yourFileOrURL);

答案 1 :(得分:0)

我已为Spark会话添加了jar文件。 错误已清除。 https://jar-download.com/?search_box=org.apache.spark%20spark.sql