我正在使用Azure Spark-CosmosDB连接器,它允许您直接从Spark群集访问CosmosDB节点,以便在HDINsight上使用Jupyter进行分析
我一直在遵循here所述的步骤,包括将所需的jar上传到Azure存储并执行%% configure magic来准备环境。
但是在尝试打开jar时由于I / O异常似乎总是终止(参见下面的纱线日志)
17/10/09 20:10:35 INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.io.IOException: Error accessing /mnt/resource/hadoop/yarn/local/usercache/livy/appcache/application_1507534135641_0014/container_1507534135641_0014_01_000001/azure-cosmosdb-spark-0.0.3-SNAPSHOT.jar)
17/10/09 20:10:35 ERROR ApplicationMaster: RECEIVED SIGNAL TERM
17/10/09 20:10:35 INFO ApplicationMaster: Unregistering ApplicationMaster with FAILED (diag message: User class threw exception: java.io.IOException: Error accessing /mnt/resource/hadoop/yarn/local/usercache/livy/appcache/application_1507534135641_0014/container_1507534135641_0014_01_000001/azure-cosmosdb-spark-0.0.3-SNAPSHOT.jar)`
不确定这是否与jar没有被复制到工作节点有关。
任何想法?谢谢,尼克