客户端扫描HDFS集群的内存并报告消耗的空间百分比

Question

我是Hadoop的新手并且得到了

的要求

编写扫描HDFS群集内存的客户端程序，并报告消耗的可用空间百分比。

任何帮助都将受到赞赏，因为我对Hadoop Stuffs非常陌生。

此致 拉胡

Answer 1

您可以使用hadoop dfsadmin -report获取有关群集的所有统计信息。输出将是这样的：

hadoop dfsadmin -report

Configured Capacity: 30224303030272 (27.49 TB)
Present Capacity: 28080468025269 (25.54 TB)
DFS Remaining: 11862609379328 (10.79 TB)
DFS Used: 16217858645941 (14.75 TB)
DFS Used%: 57.75%
Under replicated blocks: 47
Blocks with corrupt replicas: 63
Missing blocks: 47

-------------------------------------------------
Live datanodes (9):

Name: 111.111.111.111:50010
Hostname: $hostname
Rack: /default
Decommission Status : Normal
Configured Capacity: 2240100818944 (2.04 TB)
DFS Used: 1258469347618 (1.14 TB)
Non DFS Used: 179970305758 (167.61 GB)
DFS Remaining: 801661165568 (746.61 GB)
DFS Used%: 56.18%
DFS Remaining%: 35.79%
Configured Cache Capacity: 4294967296 (4 GB)
Cache Used: 0 (0 B)
Cache Remaining: 4294967296 (4 GB)
Cache Used%: 0.00%
Cache Remaining%: 100.00%
Last contact: Fri Dec 12 12:36:32 IST 2014