Question

我使用Spark MLlib的DecisionTreeModel进行预测。

 def predict(sc: SparkContext) : (Row => Row) = {
      val model = DecisionTreeModel.load(sc,"hdfs://invlxiisqa09.informatica.com:8020/user/cloudqa/anuja/myDecisionTreeClassificationModel")
     (str:Row)=> {
      val mldata5 = Vectors.dense( str.get(0).asInstanceOf[Int],str.get(1).asInstanceOf[Int],  str.get(2).asInstanceOf[Int] ,  str.get(3).asInstanceOf[Int], str.get(4).asInstanceOf[Int],  str.get(5).asInstanceOf[Int], str.get(6).asInstanceOf[Int], str.get(7).asInstanceOf[Int])
      val prediction = Row.fromSeq(String.valueOf(model.predict(mldata5)).toSeq)
      prediction
     }        
}

当我发送以下数据时

[12.0,0.0,1620.0,1850.0,6.0,270.0,234.0,270.0]

我在创建一行时得到ClassCastException。

有人能指出我错在哪里吗？

Stacktrace是

java.lang.ClassCastException: [C cannot be cast to [Ljava.lang.Object;
at org.apache.spark.sql.Row$.fromSeq(Row.scala:53)
at com.informatica.bootstrap.InfaStreaming$$anonfun$predicts$1.apply(InfaStreaming.scala:256)
at com.informatica.bootstrap.InfaStreaming$$anonfun$predicts$1.apply(InfaStreaming.scala:253)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at com.informatica.bootstrap.InfaStreaming$$anonfun$jsonPayloadGenerator$1$$anon$3.next(InfaStreaming.scala:946)
at com.informatica.bootstrap.InfaStreaming$$anonfun$jsonPayloadGenerator$1$$anon$3.next(InfaStreaming.scala:943)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.convert.Wrappers$IteratorWrapper.next(Wrappers.scala:31)
at com.informatica.adapter.streaming.kafka.common.KafkaOperations.sendMessagesBinary(KafkaOperations.java:308)
at com.informatica.bootstrap.InfaStreaming$$anonfun$writeToKafka$2$$anonfun$apply$26.apply(InfaStreaming.scala:274)
at com.informatica.bootstrap.InfaStreaming$$anonfun$writeToKafka$2$$anonfun$apply$26.apply(InfaStreaming.scala:272)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:925)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:925)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Answer 1

Java ClassCastException

Java ClassCastException是一种异常，当您尝试将类从一种类型不正确地转换为另一种类型时可能会发生此异常。 ...因为String不能直接转换为Integer - Integer不是一种String类型 - 在java中抛出ClassCastException。

Answer 2

将String转换为整数：

def tryToInt（s：String）= Try（s.toInt）.toOption

在您的代码中可能会尝试这样申请......

val prediction = Row.fromSeq（tryToInt（s：String）= Try（（model.predict（mldata5.toInt））。toSeq）

Answer 3

您的行中的某些内容不是Int，当您尝试将其强制转换为int时，它会失败。要进行故障排除，我会尝试打印出行中的所有内容并查看其中的内容。有时，当您加载数据时，列编组的类型可能会令人惊讶。

Answer 4

我遇到了同样的问题，它对我有用：

尝试

val prediction = Row(String.valueOf(model.p....

代替

val prediction = Row.fromSeq(String.valueOf(model.p....

ClassCastException [C不能转换为[Ljava.lang.Object;

4 个答案: