如何在mapreduce

Question

请告诉我如何将输出fileformat（part-r-00000）（默认文件格式）更新为另一种文件格式，如csv或txt文件格式的map reduce程序。

Answer 1

You could do this:

hdfs dfs -cat /path/in/hdfs/part* |hdfs dfs -put - /chosen/path/in/hdfs/name_of_file.txt

OR

hdfs dfs -cat /path/in/hdfs/part* |hdfs dfs -put - chosen/path/in/hdfs/name_of_file.csv

Another method is -getmerge which copies to local but then you need to -copyFromLocal back to hdfs but it serves the purpose of changing your file format:

hdfs dfs -getmerge /path/in/hdfs/part* /path/in/local/file_name.format

hdfs dfs -copyFromLocal /path/in/local/file_name.format /path/in/hdfs/archive/

Answer 2

一种方法是使用hadoop的put命令将part-r-00000文件复制到xyz.txt文件。 比如hdfs dfs -put part-r-00000到xyz.txt