Question

我们成功地从twitter提取了数据，但无法使用flume将其保存在系统中。请您解释一下

Answer 1

you might have problem in channel or sink may be that's why u r data is not storing in hdfs.

try to understan  this one

TwitterAgent.sinks.HDFS.channel = MemChannel
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://yourIP:8020/user/flume/tweets/%Y/%M/%d/%H/
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000

and chek with jps if your data node and namenode is working

无法将Twitter数据存储在水槽中

1 个答案: