Spark S3 SocketTimeoutException

时间:2015-05-06 21:07:29

标签: amazon-s3 apache-spark

我的spark应用程序在尝试拉下S3文件时遇到套接字超时异常。它在我的IDE中作为Java'main'应用程序工作正常,但在本地计算机上以独立模式启动./spark-submit脚本时会出现类似错误。

15/05/06 14:02:33 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times;
aborting job 

Exception in thread "main" org.apache.spark.SparkException:
Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most 
recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): 

java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)

在独立模式下,此代码是否在Driver / Worker中运行?是否有我应该研究的日志?

谢谢, 兰吉特

0 个答案:

没有答案