目前我正在部署EMR群集版本5.4.0，当我尝试添加一个火花步骤时，它出现错误，并始终显示FAILED状态：
我已经尝试过阅读这个用于1.8.0 PATH的web更改java版本，但它不起作用。
这是stderr的日志。
这些是我们的版本：
发布标签：emr-5.4.0 Hadoop发行版：亚马逊2.7.3应用程序：Hive 2.1.1，Pig 0.16.0，Spark 2.1.0
错误细节详细信息：线程“main”中的异常org.apache.spark.SparkException：应用程序application_1507870642238_0001以失败状态完成JAR位置：command-runner.jar主类：无参数：spark-submit --deploy-mode cluster s3：//mach-big-data-infra/ingest.py失败时的行动：继续
7/10/13 05:22:45 INFO RMProxy: Connecting to ResourceManager at ip-10-5-0-147.ec2.internal/10.5.0.147:8032
17/10/13 05:22:46 INFO Client: Requesting a new application from cluster with 1 NodeManagers
17/10/13 05:22:46 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2048 MB per container)
17/10/13 05:22:46 INFO Client: Will allocate AM container, with 1408 MB memory including 384 MB overhead
17/10/13 05:22:46 INFO Client: Setting up container launch context for our AM
17/10/13 05:22:46 INFO Client: Setting up the launch environment for our AM container
17/10/13 05:22:46 INFO Client: Preparing resources for our AM container
17/10/13 05:22:53 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
17/10/13 05:23:02 INFO Client: Uploading resource file:/mnt/tmp/spark-fdeecf22-10a0-4d51-9e2f-ff63f3787b82/__spark_libs__6942539995783678180.zip -> hdfs://ip-10-5-0-147.ec2.internal:8020/user/hadoop/.sparkStaging/application_1507870642238_0001/__spark_libs__6942539995783678180.zip
17/10/13 05:23:10 INFO Client: Uploading resource file:/etc/spark/conf/hive-site.xml -> hdfs://ip-10-5-0-147.ec2.internal:8020/user/hadoop/.sparkStaging/application_1507870642238_0001/hive-site.xml
17/10/13 05:23:17 INFO Client: Uploading resource s3://mach-big-data-infra/ingest.py -> hdfs://ip-10-5-0-147.ec2.internal:8020/user/hadoop/.sparkStaging/application_1507870642238_0001/ingest.py
17/10/13 05:23:17 INFO S3NativeFileSystem: Opening 's3://mach-big-data-infra/ingest.py' for reading
17/10/13 05:23:17 INFO Client: Uploading resource file:/usr/lib/spark/python/lib/pyspark.zip -> hdfs://ip-10-5-0-147.ec2.internal:8020/user/hadoop/.sparkStaging/application_1507870642238_0001/pyspark.zip
17/10/13 05:23:17 INFO Client: Uploading resource file:/usr/lib/spark/python/lib/py4j-0.10.4-src.zip -> hdfs://ip-10-5-0-147.ec2.internal:8020/user/hadoop/.sparkStaging/application_1507870642238_0001/py4j-0.10.4-src.zip
17/10/13 05:23:17 INFO Client: Uploading resource file:/mnt/tmp/spark-fdeecf22-10a0-4d51-9e2f-ff63f3787b82/__spark_conf__7536775293554442846.zip -> hdfs://ip-10-5-0-147.ec2.internal:8020/user/hadoop/.sparkStaging/application_1507870642238_0001/__spark_conf__.zip
17/10/13 05:23:18 INFO SecurityManager: Changing view acls to: hadoop
17/10/13 05:23:18 INFO SecurityManager: Changing modify acls to: hadoop
17/10/13 05:23:18 INFO SecurityManager: Changing view acls groups to:
17/10/13 05:23:18 INFO SecurityManager: Changing modify acls groups to:
17/10/13 05:23:18 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); groups with view permissions: Set(); users with modify permissions: Set(hadoop); groups with modify permissions: Set()
17/10/13 05:23:18 INFO Client: Submitting application application_1507870642238_0001 to ResourceManager
17/10/13 05:23:18 INFO YarnClientImpl: Submitted application application_1507870642238_0001
17/10/13 05:23:20 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:20 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1507872198394
final status: UNDEFINED
tracking URL: http://ip-10-5-0-147.ec2.internal:20888/proxy/application_1507870642238_0001/
user: hadoop
17/10/13 05:23:21 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:22 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:23 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:24 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:25 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:26 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:27 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:28 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:29 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:30 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:31 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:32 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:33 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:34 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:35 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:36 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:37 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:38 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:39 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:40 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:41 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:42 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:43 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:44 INFO Client: Application report for application_1507870642238_0001 (state: ACCEPTED)
17/10/13 05:23:45 INFO Client: Application report for application_1507870642238_0001 (state: RUNNING)
17/10/13 05:23:45 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.5.0.9
ApplicationMaster RPC port: 0
queue: default
start time: 1507872198394
final status: UNDEFINED
tracking URL: http://ip-10-5-0-147.ec2.internal:20888/proxy/application_1507870642238_0001/
user: hadoop
17/10/13 05:23:46 INFO Client: Application report for application_1507870642238_0001 (state: RUNNING)
17/10/13 05:23:47 INFO Client: Application report for application_1507870642238_0001 (state: RUNNING)
17/10/13 05:23:48 INFO Client: Application report for application_1507870642238_0001 (state: RUNNING)
17/10/13 05:23:49 INFO Client: Application report for application_1507870642238_0001 (state: RUNNING)
17/10/13 05:23:50 INFO Client: Application report for application_1507870642238_0001 (state: RUNNING)