通过AWS CLI将Flink自定义JAR提交到AWS EMR集群时出错

时间:2018-10-30 14:13:42

标签: apache-flink amazon-emr

我在AWS EMR内运行一个Flink集群。我正在尝试使用AWS CLI将我的批处理jar文件部署到集群中,作为步骤功能的一部分。批处理jar文件的参数位于s3中。但是,当我提交命令时,它已成功提交,并且在执行时它会返回一些错误消息。

但是,当我登录到主节点并提交作业时,它运行良好。 这就是我提交工作的方式

flink run -m yarn-cluster -yn 2 -c com.test.MyJob /home/hadoop/flinktest/batch.jar --flink.props.file /home/hadoop/flinktest/flink.props --common.props.file /home/hadoop/flinktest/common.props --job.props.file /home/hadoop/flinktest/testjob.props  --date 20181024

AWS CLI命令失败:-

aws emr add-steps --cluster-id j-XXXXXX --steps Type=CUSTOM_JAR,Name=testjob,ActionOnFailure=CONTINUE,Jar=s3://mybucket/flinkjob/batch.jar,MainClass=com.test.MyJob,Args="flink","run","-m","yarn-cluster","-yn","2","--flink.props.file","s3://mybucket/flinkjob/flink.props","--common.props.file","s3://mybucket/flinkjob/common.props","--job.props.file","s3://mybucket/flinkjob/testjob.props","--date","20181022"

--

aws emr add-steps --cluster-id j-XXXXXX --steps Type=CUSTOM_JAR,Name=testjob,ActionOnFailure=CONTINUE,Jar=s3://mybucket/flinkjob/batch.jar,Args=["flink","run","-m","yarn-cluster","-yn","2","-c","com.test.MyJob","--flink.props.file","s3://mybucket/flinkjob/flink.props","--common.props.file","s3://mybucket/flinkjob/common.props","--job.props.file","s3://mybucket/flinkjob/testjob.props","--date","20181022"]

我在这里做什么错了?

0 个答案:

没有答案