Question

我正在使用Azure数据块，并且具有已挂载的BLOB存储。我尝试使用以下方式将数据框保存到其中：

df.write.mode("overwrite").format("com.databricks.spark.csv").option("header","true").csv("/mnt/gl")

我收到以下错误：

 shaded.databricks.org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.

---------------------------------------------------------------------------
Py4JJavaError                             Traceback (most recent call last)
<command-1181559690773266> in <module>()
----> 1 df.write.mode("overwrite").format("com.databricks.spark.csv").option("header","true").csv("/mnt/gl")

/databricks/spark/python/pyspark/sql/readwriter.py in csv(self, path, mode, compression, sep, quote, escape, header, nullValue, escapeQuotes, quoteAll, dateFormat, timestampFormat, ignoreLeadingWhiteSpace, ignoreTrailingWhiteSpace, charToEscapeQuoteEscaping, encoding, emptyValue)
    927                        charToEscapeQuoteEscaping=charToEscapeQuoteEscaping,
    928                        encoding=encoding, emptyValue=emptyValue)
--> 929         self._jwrite.csv(path)
    930 
    931     @since(1.5)

/databricks/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in __call__(self, *args)
   1255         answer = self.gateway_client.send_command(command)
   1256         return_value = get_return_value(
-> 1257             answer, self.gateway_client, self.target_id, self.name)
   1258 
   1259         for temp_arg in temp_args:

/databricks/spark/python/pyspark/sql/utils.py in deco(*a, **kw)

更新：重新创建SAS密钥后，错误现在显示“作业已中止”

Answer 1

尝试将“ csv”更改为“保存”：

df \
.write \
.mode("overwrite") \
.format("com.databricks.spark.csv") \
.option("header","true") \
.save("/mnt/gl")

将Databricks数据帧写入BLOB存储

1 个答案: