Mapreduce日志 - 文件与HDFS

时间:2016-01-07 22:22:08

标签: hadoop

我提交了mapreduce,这是我的输出。

下面给出的FILE和HDFS有什么区别?

16/01/07 21:49:58 INFO mapreduce.Job: Counters: 38
    File System Counters
        FILE: Number of bytes read=4011012
        FILE: Number of bytes written=8400605
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=11928267
        HDFS: Number of bytes written=883509
        HDFS: Number of read operations=37
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=6

1 个答案:

答案 0 :(得分:2)

FILE - 给出在中间文件上执行的I / O量,这些内部文件在map和reduce阶段(sort& shuffle阶段)内部维护

HDFS - 映射器读取的数据量和reducer写入的数据。