在hortonworks沙箱vm中创建水槽代理,用于将天气数据流式传输到HDFS

时间:2015-03-17 06:28:37

标签: hdfs flume data-stream

我正在配置水槽来传输天气数据, 我已经写了下面给出的水槽剂,

WeatherAgent.sources =天气

WeatherAgent.sinks = hdfs-write

WeatherAgent.channels = memory-channel

WeatherAgent.sources.Weather.type =天气

WeatherAgent.sources.Weather.bind = api.openweathermap.org/data/2.5/forecast/city?id=524901&APPID = *********

WeatherAgent.sources.Weather.port = 11111

WeatherAgent.sinks.hdfs-write.type = hdfs

WeatherAgent.sinks.hdfs-write.hdfs.path = hdfs:// localhost:8020 / user / hadoop / flume

WeatherAgent.sinks.hdfs-write.rollInterval = 30

WeatherAgent.sinks.hdfs-write.hdfs.writeFormat =文本

WeatherAgent.sinks.hdfs-write.hdfs.fileType =的数据流中

WeatherAgent.channels.memoryChannel.type = memory

WeatherAgent.channels.memoryChannel.capacity = 10000

WeatherAgent.sources.Weather.channels = memoryChannel

WeatherAgent.sinks.hdfs-write.channel = memoryChannel

我收到错误

  

天气代理商不包含任何有效渠道,使其无效。

我是新手,我不知道在代理配置文件中添加了哪些参数,我甚至不知道书面代理是正确还是错误,请帮我解决这个问题。 如果有任何帮助链接可以编写自定义或新代理,请发布。

提前致谢

Suyog

1 个答案:

答案 0 :(得分:0)

 it is type mismatching. In your configuration file you mentioned  

    WeatherAgent.channels= memory-channel

  but you are using  memoryChannel instead of memory-channel

    WeatherAgent.channels.memoryChannel.type = memory

    WeatherAgent.channels.memoryChannel.capacity=10000

    WeatherAgent.sources.Weather.channels=memoryChannel