使用GenericOptionsParser和ToolRunner的分布式文件

时间:2014-12-09 16:22:28

标签: hadoop distributed-caching toolrunner

我曾经使用(使用GenericOptionsParser)运行hadoop mapreduce

hadoop jar app.jar app -files /home/some/file#file

然而,在最近版本的hadoop中我得到了

Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.

那么如何使用ToolRunner实现-files选项在所有mapreduce节点之间共享?

1 个答案:

答案 0 :(得分:0)

显然以前的已被弃用......

这就是我做的......

import org.apache.hadoop.conf.Configuration; . . . Configuration conf = new Configuration(); . Job job = Job.getInstance(conf, "appname"); job.setJarByClass(appname.class); . . job.addCacheFile(new URI("/file/in/hdfs/APP/appfile#appfile")); . .