Question

我尝试使用sparksession createDataframe函数，我不断收到错误。 xxx不是字符串模式的有效外部类型。 xxx可以是Arrays $ ArrayList和Ljava.lang.String。

这是创建行的代码

List<StructField> structfields = new ArrayList<StructField>();
for(ColumnMetaData c : metadata){
            System.out.println(c.getColumnName());
            structfields.add(DataTypes.createStructField( c.getColumnName(), DataTypes.StringType, false ));

        }

        StructType schema = DataTypes.createStructType(structfields);

我制作这样的架构

JavaRDD<Row> rdd = sc.parallelize(all_rows);
Dataset<Row> new_ds = spark.createDataFrame(rdd,schema);

我创建了数据集

staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, l_linestatus), StringType), true) AS l_linestatus#469
+- staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, l_linestatus), StringType), true)
   +- validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, l_linestatus), StringType)
      +- getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, l_linestatus)
         +- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)
            +- input[0, org.apache.spark.sql.Row, true]

我确实参考了this，但似乎答案对我不起作用。

还有来自spark的日志

    angular
      .module('app', [])
      .controller('ctrl', function($scope) {
        $scope.disabled = true;
      });
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.23/angular.min.js"></script>

    <style>
      .ng-disabled {
        background: #E9B96E;
      }
    </style>

    <span ng-app="app" ng-controller="ctrl">
    <input type="file" ng-class="{'ng-disabled': disabled}" ng-disabled="disabled" id="input1" />
    <input type="file" ng-class="{'ng-disabled': disabled}" ng-disabled="disabled" id="input2"/>
    </span>

xxx不是字符串模式的有效外部类型

0 个答案: