我在哪里可以找到cloudera cdh 5.10中的Hadoop日志以及如何使用Flume将这些(Hadoop节点日志)加载到HDFS中

时间:2017-12-04 20:29:47

标签: hadoop flume cloudera-cdh flume-ng

到目前为止我能找到的就是这个,在/ var / log / hadoop-yarn目录下我可以看到以下.out和.log文件。 但我不确定这些是否实际上被称为hadoop日志文件,我需要加载到hdfs。如果这些是我们所说的hadoop日志文件时的文件,那么我的.conf文件看起来能够成功加载到hdfs中。

[cloudera@quickstart /]$ cd /var/log/hadoop-yarn
[cloudera@quickstart hadoop-yarn]$ ls -lart
total 2684
-rw-r--r--   1 yarn yarn       704 Nov  3 12:40 yarn-yarn-nodemanager-quickstart.cloudera.out.2
-rw-r--r--   1 yarn yarn       704 Nov  3 12:40 yarn-yarn-proxyserver-quickstart.cloudera.out.2
-rw-r--r--   1 yarn yarn       704 Nov  3 12:40 yarn-yarn-resourcemanager-quickstart.cloudera.out.2
-rw-r--r--   1 yarn yarn       704 Nov  4 01:40 yarn-yarn-nodemanager-quickstart.cloudera.out.1
-rw-r--r--   1 yarn yarn       704 Nov  4 01:40 yarn-yarn-proxyserver-quickstart.cloudera.out.1
-rw-r--r--   1 yarn yarn       704 Nov  4 01:40 yarn-yarn-resourcemanager-quickstart.cloudera.out.1
-rw-r--r--   1 yarn yarn       704 Nov  4 01:50 yarn-yarn-nodemanager-quickstart.cloudera.out
-rw-r--r--   1 yarn yarn       704 Nov  4 01:50 yarn-yarn-proxyserver-quickstart.cloudera.out
-rw-r--r--   1 yarn yarn     72615 Nov  4 01:50 yarn-yarn-proxyserver-quickstart.cloudera.log
-rw-r--r--   1 yarn yarn       704 Nov  4 01:50 yarn-yarn-resourcemanager-quickstart.cloudera.out
drwxr-xr-x   2 yarn yarn      4096 Nov  4 01:52 containers
-rw-r--r--   1 yarn yarn    116956 Nov  4 01:54 yarn-yarn-resourcemanager-quickstart.cloudera.log
-rw-r--r--   1 yarn yarn    105950 Nov  4 01:54 yarn-yarn-nodemanager-quickstart.cloudera.log
drwxr-xr-x   2 yarn hadoop    4096 Nov  4 10:28 stacks
drwxrwxr-x   5 yarn hadoop    4096 Nov  4 10:28 .
drwxr-xr-x. 41 root root      4096 Dec  4 08:46 ..
-rw-r--r--   1 yarn yarn   1710159 Dec  4 10:14 hadoop-cmf-yarn-NODEMANAGER-quickstart.cloudera.log.out
drwxr-xr-x   2 yarn hadoop    4096 Dec  4 10:46 container
-rw-r--r--   1 yarn yarn    671383 Dec  4 10:46 hadoop-cmf-yarn-RESOURCEMANAGER-quickstart.cloudera.log.out
[cloudera@quickstart hadoop-yarn]$

0 个答案:

没有答案