向通道写入更多事件会导致通道完全异常

时间:2015-04-21 07:01:23

标签: flume-ng

我正在使用flume JMS源将来自ActiveMQ的消息出列,并使用自定义转换器将此消息转换为List<Event>

频道配置

agent.channels.c1.type = memory
agent.channels.c1.capacity = 1000000
agent.channels.c1.transactionCapacity = 1500

List<Event>的大小小于或等于1500(渠道交易容量)时,水槽会将事件写入渠道,但如果事件大小大于1500,那么我得到以下异常< / p>

错误日志

21 Apr 2015 12:19:28,245 WARN  [PollableSourceRunner-JMSSource-s1] (org.apache.flume.source.jms.JMSSource.doProcess:263)  - Error appending event to channel. Channel might be full. Consider increasing the channel capacity or make sure the sinks perform faster.
org.apache.flume.ChannelException: Unable to put batch on required channel: org.apache.flume.channel.MemoryChannel{name: c1}
    at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
    at org.apache.flume.source.jms.JMSSource.doProcess(JMSSource.java:257)
    at org.apache.flume.source.AbstractPollableSource.process(AbstractPollableSource.java:54)
    at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
    at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.flume.ChannelException: Put queue for MemoryTransaction of capacity 1500 full, consider committing more frequently, increasing capacity or increasing thread count
    at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doPut(MemoryChannel.java:84)
    at org.apache.flume.channel.BasicTransactionSemantics.put(BasicTransactionSemantics.java:93)
    at org.apache.flume.channel.BasicChannelSemantics.put(BasicChannelSemantics.java:80)
    at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:189)
    ... 4 more

如何解决这个问题?

注意:事件大小根据ActiveMQ消息动态变化

0 个答案:

没有答案