将数据帧写入Blob存储-文件为空或作业中止

时间:2019-03-28 21:59:52

标签: azure-databricks

我正在尝试编写一个包含某些字符串类型的简单数据框作为csv,并将其存储在我的Blob存储容器中。我已使用具有读取和写入权限且未过期的SAS将blob容器安装在工作区中。我还希望它是一个文件,因此使用了coalesce(1),但是要以PART-1,PART-0的形式出现。

这是我的代码

file_path_blob = "/mnt/LogData/predictions1"
anomalies_df.coalesce(1).write.csv(file_path_blob)

当我运行作业时,它写了一些东西,但最终失败了,并显示了错误消息,如

Caused by: com.microsoft.azure.storage.StorageException: This request is not authorized to perform this operation using this permission.
    at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:89)
    at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305)
    at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:178)
    at com.microsoft.azure.storage.blob.CloudBlob.delete(CloudBlob.java:928)
    at shaded.databricks.org.apache.hadoop.fs.azure.StorageInterfaceImpl$CloudBlobWrapperImpl.delete(StorageInterfaceImpl.java:313)
    at shaded.databricks.org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.safeDelete(AzureNativeFileSystemStore.java:2533)
    at shaded.databricks.org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.rename(AzureNativeFileSystemStore.java:2727)
    ... 28 more

enter image description here

enter image description here

0 个答案:

没有答案
相关问题