Question

我有3个特工。 1使用内存通道在Windows上运行，其他2个在Linux上使用文件通道从Windows代理获取数据并放入Hbase。

任何人都可以建议为什么发生以下错误以及阻止它的步骤是什么？

2013-12-23 14:50:15,290 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG - org.apache.flume.sink.AvroSink.destroyConnection(AvroSink.java:199)] Avro sinksink1 closing avro client: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }
2013-12-23 14:50:15,290 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event.
Exception follows.org.apache.flume.EventDeliveryException: Failed to send events
        at org.apache.flume.sink.AvroSink.process(AvroSink.java:325)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }: Failed to send batch
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:236)
        at org.apache.flume.sink.AvroSink.process(AvroSink.java:309)        ... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }: Handshake timed out after 20000ms
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:280)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:224)    ... 4 more
Caused by: java.util.concurrent.TimeoutException
        at java.util.concurrent.FutureTask.get(Unknown Source)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:278)

以下是我的Windows配置文件

a1.sources = source1
a1.channels = channel1 channel2
a1.sinks = sink1 sink2

a1.sources.source1.handler = com.flume.handler.DynamicJSONHandler
a1.sources.source1.type = org.apache.flume.source.http.HTTPSource
a1.sources.source1.bind = 192.168.101.29
a1.sources.source1.port = 2001

a1.channels.channel1.type = org.apache.flume.channel.MemoryChannel
a1.channels.channel1.capacity = 1000
a1.channels.channel1.transactionCapacity = 1000

a1.sinks.sink1.type = org.apache.flume.sink.AvroSink
a1.sinks.sink1.hostname = 192.168.101.232
a1.sinks.sink1.port = 3001

a1.channels.channel2.type = org.apache.flume.channel.MemoryChannel
a1.channels.channel2.capacity = 1000
a1.channels.channel2.transactionCapacity = 1000

a1.sinks.sink2.type = org.apache.flume.sink.AvroSink
a1.sinks.sink2.hostname = 192.168.101.233
a1.sinks.sink2.port = 3001

a1.sources.source1.channels = channel1 channel2
a1.sinks.sink1.channel = channel1
a1.sinks.sink2.channel = channel2

Answer 1

AvroSink旨在将数据写入另一个水槽代理，该代理有一个Avro Source监听该特定来源。 Avro Sink启动与其连接的源的握手。但看起来握手在20秒后超时 - 因此很可能您的网络面临严重的延迟问题。您是否在运行源代码的代理上看到任何问题？

Flume代理商无法提供活动

1 个答案: