MapReduce 2作业在本地运行,而不是提交给纱线

时间:2017-02-14 15:21:29

标签: java hadoop mapreduce

所以我尝试将我的作业配置为提交给yarn,而是在本地运行:

config.set("yarn.resourcemanager.address", "ADDRESS:8032");
config.set("mapreduce.framework.name", "yarn"); 
config.set("fs.default.name", "hdfs://ADDRESS:8020");

如果我设置 mapred.job.tracker ,则会失败并显示:

Exception in thread "main" org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcServerException): Unknown rpc kind in rpc headerRPC_WRITABLE

因为它不是MR1。

那么为什么应用程序没有提交给纱线?

1 个答案:

答案 0 :(得分:1)

通过这样做解决了这个问题:

config.set("yarn.resourcemanager.address", "ADDRESS:8032");
config.set("yarn.resourcemanager.scheduler.address", "ADDRESS:8030");
config.set("yarn.resourcemanager.resource-tracker.address", "ADDRESS:8031");
config.set("yarn.resourcemanager.admin.address", "ADDRESS:8033");

而不是:

config.set("yarn.resourcemanager.address", "ADDRESS:8032");