苏打API的此错误消息中我缺少什么?

时间:2020-10-02 19:20:01

标签: apache-spark pyspark soda

stackoverflow的大家好!

我有一个汽水api问题尚未得到回答?好吧,我猜我没有找到我想要的方式。我一直在构建此API以获取一些数据,但在进行调用时似乎失败。我具有所需的Jar文件,并且正在集群中读取它。

File "spark_app.py", line 205, in <module>
    sodasDf = spark.read.format('org.apache.dsext.spark.datasource.rest.RestDataSource').options(**prms_Soda).load()
  File "/app/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 172, in load
  File "/app/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
  File "/app/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
  File "/app/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o109.load.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 9.0 failed 4 times, most recent failure: Lost task 0.3 in stage 9.0 (TID 12, cvglpcdh13.td.afg, executor 4): java.lang.NullPointerException
        at org.apache.dsext.spark.datasource.rest.RESTRelation$$anonfun$3.apply(RestRelation.scala:108)
        at org.apache.dsext.spark.datasource.rest.RESTRelation$$anonfun$3.apply(RestRelation.scala:108)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
        at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
        at org.apache.dsext.spark.datasource.rest.RESTRelation.org$apache$dsext$spark$datasource$rest$RESTRelation$$callRest(RestRelation.scala:108)
        at org.apache.dsext.spark.datasource.rest.RESTRelation$$anonfun$2.apply(RestRelation.scala:100)
        at org.apache.dsext.spark.datasource.rest.RESTRelation$$anonfun$2.apply(RestRelation.scala:100)
        at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
        at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
        at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
        at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$11$$anon$1.hasNext(WholeStageCodegenExec.scala:624)
        at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
        at scala.collection.Iterator$class.isEmpty(Iterator.scala:331)
        at scala.collection.AbstractIterator.isEmpty(Iterator.scala:1334)
        at scala.collection.TraversableOnce$class.reduceLeftOption(TraversableOnce.scala:203)
        at scala.collection.AbstractIterator.reduceLeftOption(Iterator.scala:1334)
        at scala.collection.TraversableOnce$class.reduceOption(TraversableOnce.scala:210)
        at scala.collection.AbstractIterator.reduceOption(Iterator.scala:1334)

我不确定这里缺少什么。看来有人早些时候曾问过相同的问题,所以(Can someone help me with this error message:),我不确定他们和我的是否相同,或者我遇到了另一个错误,但我们将不胜感激

0 个答案:

没有答案