我尝试运行数据管道作业,但EmrActivity步骤已达到FAILED状态,但没有错误代码或错误消息:
名称: @EMR集群执行work_2013-09-03T16:15:00 查看实例字段 描述: 最新尝试次数:3,尝试左:0 选择此实例的尝试:
状态: 失败 错误代码: 错误讯息:
任何想法为什么?我在哪里可以找到有关潜在问题的更多信息?
工作很简单:启动EMR集群并运行猪脚本(其中xxx是我的存储桶名称):
{ "objects": [ { "id":"Default", "failureAndRerunMode":"cascade" }, { "id" : "MyScheduleID", "type" : "Schedule", "period" : "1 hour", "startDateTime" : "2013-09-03T19:00:00", "endDateTime" : "2013-09-03T20:00:00" }, { "id" : "MyEmrCluster", "name" : "EMR cluster to perform the work", "type" : "EmrCluster", "hadoopVersion" : "0.20", "masterInstanceType" : "m1.small", "coreInstanceType" : "m1.medium", "coreInstanceCount" : "2", "terminateAfter": "1 Hours", "schedule": { "ref": "MyScheduleID" }, "logUri":"s3://xxx/amazonlogs", "emrLogUri":"s3://xxx/amazonlogs" }, { "id" : "MyEmrActivity", "name" : "Work to perform on my data", "type" : "EmrActivity", "runsOn" : {"ref" : "MyEmrCluster"}, "schedule": { "ref": "MyScheduleID" }, "step": "s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar,s3://us-east-1.elasticmapreduce/libs/pig/pig-script,--base-path,s3://us-east-1.elasticmapreduce/libs/pig/,--install-pig,--pig-versions,latest", "step": "s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar,s3://us-east-1.elasticmapreduce/libs/pig/pig-script,--base-path,s3://us-east-1.elasticmapreduce/libs/pig/,--pig-versions,latest,--run-pig-script,--args,-f,s3://xxx/carls_minimal_script.pig" } ] }
这个配置看起来不错吗? 我在s3:// xxx / amazonlogs
中看不到任何内容答案 0 :(得分:2)
以下是您可以尝试的几件事
转到“https://console.aws.amazon.com/elasticmapreduce/home”,找到已启动的相应群集(基于时间戳),单击“调试”,您应找到有关每个步骤的日志。
或者从AWS控制台启动EMR集群,登录主节点,运行Pig脚本以检查其是否正常工作。