Question

我已经用所需的数据创建了数据框，并试图写入HIVE DB。我收到此错误。 df2结构就是这样

  root
 |-- hashval: string (nullable = true)
 |-- ftr_[0]: string (nullable = true)
 |-- ftr_[1]: string (nullable = true)
 |-- ftr_[2]: string (nullable = true)
 |-- ftr_[3]: string (nullable = true)
 |-- ftr_[4]: string (nullable = true)
 .
 .
 . 
 more

当我尝试写入数据库时出现此错误。不知道是什么原因造成的。数据不为空。我知道。

df2.registerTempTable("test")
spark.sql("DROP TABLE IF EXISTS  scdb.spec_test")
spark.sql("CREATE TABLE IF NOT EXISTS scdb.spec_test  as select * from test" )

这是错误

at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 77.0 
failed 4 times, most recent failure: Lost task 3.3 in stage 77.0 (TID 1614, hpcb04xxxx.hpc.xxx.com, 
executor 155): java.util.concurrent.ExecutionException: 
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 1220, Column 14: failed 
to dompile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 1220, Column 
 14: Expression "isNull_6" is not an rvalue     at org.spark_project.guava.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:552)

在Pyspark中写入HIve DB时暂存故障

0 个答案: