我试图将数据帧写入cassandra表,但遇到错误......我找不到解决方法

时间:2019-07-22 10:27:03

标签: pyspark

回溯(最近通话最近一次):

  

文件“ py_spark.py”,第235行,在       .options(table =“ struct_str”,keyspace =“ acrs”)\文件“ /usr/lib/python3.6/site-packages/pyspark/sql/readwriter.py”,行   732,保存中       self._jwrite.save()文件“ /usr/lib/python3.6/site-packages/py4j/java_gateway.py”,第1257行,在   致电       答案,self.gateway_client,self.target_id,self.name)文件“ /usr/lib/python3.6/site-packages/pyspark/sql/utils.py”,第63行,在   装饰       在第328行中返回f(* a,** kw)文件“ /usr/lib/python3.6/site-packages/py4j/protocol.py”   get_return_value       格式(target_id,“。”,名称),值)py4j.protocol.Py4JJavaError:调用o654.save时发生错误。 :   org.apache.spark.SparkException:由于阶段失败,作业中止了:   阶段1.0中的任务0失败1次,最近一次失败:任务0.0丢失   在阶段1.0(TID 8,本地主机,执行程序驱动程序)中:   com.datastax.spark.connector.types.TypeConversionException:无法   将类型为java.lang.String的对象0x000003e7转换为   java.nio.ByteBuffer。           在com.datastax.spark.connector.types.TypeConverter $$ anonfun $ convert $ 1.apply(TypeConverter.scala:43)           在com.datastax.spark.connector.types.TypeConverter $ ByteBufferConverter $$ anonfun $ convertPF $ 11.applyOrElse(TypeConverter.scala:256)处           在com.datastax.spark.connector.types.TypeConverter $ class.convert(TypeConverter.scala:41)           在com.datastax.spark.connector.types.TypeConverter $ ByteBufferConverter $ .com $ datastax $ spark $ connector $ types $ NullableTypeConverter $$ super $ convert(TypeConverter.scala:253)           在com.datastax.spark.connector.types.NullableTypeConverter $ class.convert(TypeConverter.scala:54)           在com.datastax.spark.connector.types.TypeConverter $ ByteBufferConverter $ .convert(TypeConverter.scala:253)           在com.datastax.spark.connector.types.TypeConverter $ OptionToNullConverter $$ anonfun $ convertPF $ 36.applyOrElse(TypeConverter.scala:796)           在com.datastax.spark.connector.types.TypeConverter $ class.convert(TypeConverter.scala:41)           在com.datastax.spark.connector.types.TypeConverter $ OptionToNullConverter.com $ datastax $ spark $ connector $ types $ NullableTypeConverter $$ super $ convert(TypeConverter.scala:779)           在com.datastax.spark.connector.types.NullableTypeConverter $ class.convert(TypeConverter.scala:54)           在com.datastax.spark.connector.types.TypeConverter $ OptionToNullConverter.convert(TypeConverter.scala:779)           在com.datastax.spark.connector.writer.SqlRowWriter $$ anonfun $ readColumnValues $ 1.apply $ mcVI $ sp(SqlRowWriter.scala:26)处           在scala.collection.immutable.Range.foreach $ mVc $ sp(Range.scala:160)           在com.datastax.spark.connector.writer.SqlRowWriter.readColumnValues(SqlRowWriter.scala:24)           在com.datastax.spark.connector.writer.SqlRowWriter.readColumnValues(SqlRowWriter.scala:12)           在com.datastax.spark.connector.writer.BoundStatementBuilder.bind(BoundStatementBuilder.scala:99)           在com.datastax.spark.connector.writer.GroupingBatchBuilder.next(GroupingBatchBuilder.scala:106)           在com.datastax.spark.connector.writer.GroupingBatchBuilder.next(GroupingBatchBuilder.scala:31)           在scala.collection.Iterator $ class.foreach(Iterator.scala:891)           在com.datastax.spark.connector.writer.GroupingBatchBuilder.foreach(GroupingBatchBuilder.scala:31)           在com.datastax.spark.connector.writer.TableWriter $$ anonfun $ writeInternal $ 1.apply(TableWriter.scala:233)           在com.datastax.spark.connector.writer.TableWriter $$ anonfun $ writeInternal $ 1.apply(TableWriter.scala:210)           在com.datastax.spark.connector.cql.CassandraConnector $$ anonfun $ withSessionDo $ 1.apply(CassandraConnector.scala:112)           在com.datastax.spark.connector.cql.CassandraConnector $$ anonfun $ withSessionDo $ 1.apply(CassandraConnector.scala:111)           在com.datastax.spark.connector.cql.CassandraConnector.closeResourceAfterUse(CassandraConnector.scala:145)           在com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:111)           在com.datastax.spark.connector.writer.TableWriter.writeInternal(TableWriter.scala:210)           在com.datastax.spark.connector.writer.TableWriter.insert(TableWriter.scala:197)           在com.datastax.spark.connector.writer.TableWriter.write(TableWriter.scala:183)           在com.datastax.spark.connector.RDDFunctions $$ anonfun $ saveToCassandra $ 1.apply(RDDFunctions.scala:36)           在com.datastax.spark.connector.RDDFunctions $$ anonfun $ saveToCassandra $ 1.apply(RDDFunctions.scala:36)           在org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)           在org.apache.spark.scheduler.Task.run(Task.scala:121)           在org.apache.spark.executor.Executor $ TaskRunner $$ anonfun $ 10.apply(Executor.scala:408)           在org.apache.spark.util.Utils $ .tryWithSafeFinally(Utils.scala:1360)           在org.apache.spark.executor.Executor $ TaskRunner.run(Executor.scala:414)           在java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)           在java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:624)           在java.lang.Thread.run(Thread.java:748)

0 个答案:

没有答案