Question

我创建了一个带有spark和drools的方法UDF，当我在eclipse上运行java side时没有问题，但是当我在drools side上运行时，它给了我这个转换错误：

java.lang.ClassCastException：无法在组织实例中将scala.collection.immutable.List $ SerializationProxy的实例分配给scala.collection.Seq类型的字段org.apache.spark.sql.execution.columnar.InMemoryTableScanExec.predicates类型的字段.apache.spark.sql.execution.columnar.InMemoryTableScanExec 在java.io.ObjectStreamClass $ FieldReflector.setObjFieldValues（ObjectStreamClass.java:2287）在java.io.ObjectStreamClass.setObjFieldValues（ObjectStreamClass.java:1417）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2293）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.readObject（ObjectInputStream.java:431）在scala.collection.immutable.List $ SerializationProxy.readObject（List.scala：479）在sun.reflect.GeneratedMethodAccessor11.invoke（未知来源）在sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）在java.lang.reflect.Method.invoke（Method.java:498）在java.io.ObjectStreamClass.invokeReadObject（ObjectStreamClass.java:1170）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2178）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.readObject（ObjectInputStream.java:431）在scala.collection.immutable.List $ SerializationProxy.readObject（List.scala：479）在sun.reflect.GeneratedMethodAccessor11.invoke（未知来源）在sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）在java.lang.reflect.Method.invoke（Method.java:498）在java.io.ObjectStreamClass.invokeReadObject（ObjectStreamClass.java:1170）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2178）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.defaultReadFields（ObjectInputStream.java:2287）在java.io.ObjectInputStream.readSerialData（ObjectInputStream.java:2211）在java.io.ObjectInputStream.readOrdinaryObject（ObjectInputStream.java:2069）在java.io.ObjectInputStream.readObject0（ObjectInputStream.java:1573）在java.io.ObjectInputStream.readObject（ObjectInputStream.java:431）在scala.collection.immutable.List $ SerializationProxy.readObject（List.scala：479）

禁酒规则

function Dataset categories(Dataset ds) {

StructType newSchema = new StructType()
      .add("CR", DataTypes.StringType, false)
      .add("CP", DataTypes.StringType, false);

UDF3<Integer, String, String, GenericRowWithSchema> categoryUDF = new UDF3<Integer, String, String, GenericRowWithSchema>() {

       public GenericRowWithSchema call(Integer r, String t, String q) throws Exception {
         return new GenericRowWithSchema(new String[]{"NULL", "NULL"}, newSchema);
            }
        };
 ds.sparkSession().udf().register("categoryUDF", categoryUDF, newSchema);

    return ds.withColumn("categorie",
           functions.callUDF("categoryUDF",
               functions.col("rlv_reason"),
               functions.col("rlv_type"),
               functions.col("qualification")))
       .select(ds.col("*"), functions.col("categorie.*"));
    }

rule "test UDF"
ruleflow-group "data-enrichment"
no-loop true
when
    $ds : Dataset();
    $data : Dataset() from categories($ds);
    eval (((Dataset)$data.filter("CP == 'NULL' or CR == 'NULL'")).count() > 0)
then
    System.out.println("test UDF work" );
    $data.show();
end

**执行：**

   StatelessKieSession kieSession = DroolsHelper.startStatelessKieSession();
    List<Command> cmdList = Arrays.asList(
         CommandFactory.newInsert(dataset),               
         CommandFactory.newStartProcess(processId)
    );
    kieSession.execute(CommandFactory.newBatchExecution(cmdList));

依赖项：

简单的Java Spark + Drools应用程序中的ClassCastException

0 个答案: