我想使用flume从远程HornetQ主题获取数据并将数据放入HDFS。
我有一个示例flume.conf文件,它似乎允许从远程ActiveMQ服务器读取:
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = jms
a1.sources.r1.initialContextFactory = org.apache.activemq.jndi.ActiveMQInitialContextFactory
a1.sources.r1.connectionFactory = GenericConnectionFactory
a1.sources.r1.providerURL = tcp://10.xxx.xxx.xx:7676
a1.sources.r1.destinationName = topic_name
a1.sources.r1.destinationType = TOPIC
# Describe the sink
a1.sinks.k1.type = hdfs
a1.sinks.k1.hdfs.path = hdfs://localhost:8020/data/flume/
a1.sinks.k1.hdfs.fileType = DataStream
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
特别是,这部分配置文件存在问题:
# Describe/configure the source
a1.sources.r1.type = jms
a1.sources.r1.initialContextFactory = org.apache.activemq.jndi.ActiveMQInitialContextFactory
a1.sources.r1.connectionFactory = GenericConnectionFactory
a1.sources.r1.providerURL = tcp://10.xxx.xxx.xx:7676
a1.sources.r1.destinationName = topic_name
a1.sources.r1.destinationType = TOPIC
我只在互联网上找到一些HornetQ接收插件和一些HornetQ频道插件。但我没有找到源插件。
那么,是否可以使用Flume从HornetQ读取数据?如果是,怎么样?
非常感谢。