第二层Flume代理不绑定到第一层

时间:2017-10-08 02:19:02

标签: flume flume-ng cloudera-quickstart-vm

我正在尝试使用Cloudera VM设置一个简单的两层水槽代理。 代码如下:

# First-tier agent

agent1.sources = source1 
agent1.sinks = sink1
agent1.channels = channel1

agent1.sources.source1.channels = channel1 
agent1.sinks.sink1.channel = channel1 

agent1.sources.source1.type = spooldir 
agent1.sources.source1.spoolDir = /tmp/spooldir 

agent1.sinks.sink1.type = avro 
agent1.sinks.sink1.hostname = localhost 
agent1.sinks.sink1.port = 10000

agent1.channels.channel1.type = file
agent1.channels.channel1.checkpointDir = /tmp/flume/channel1/check
agent1.channels.channel1.dataDirs = /tmp/flume/channel1/data

# Second-tier agent

agent2.sources = source2 
agent2.sinks = sink2
agent2.channels = channel2

agent2.sources.source2.channels = channel2
agent2.sinks.sink2.channel = channel2

agent2.sources.source2.type = avro 
agent2.sources.source2.bind = localhost 
agent2.sources.source2.port = 10000

agent2.sinks.sink2.type = hdfs 
agent2.sinks.sink2.hdfs.path = /tmp/flume
agent2.sinks.sink2.hdfs.filePrefix = events
agent2.sinks.sink2.hdfs.fileSuffix = .log
agent2.sinks.sink2.hdfs.fileType = DataStream

agent2.channels.channel2.type = file
agent2.channels.channel2.checkpointDir = /tmp/flume/channel2/check
agent2.channels.channel2.dataDirs = /tmp/flume/channel2/data

第一个代理看起来不错,但是一旦我运行第二个代理,它的源就无法连接到第一个代理的接收器。 它给出了以下错误:

17/10/06 14:35:33 INFO node.Application: Starting Sink sink2
17/10/06 14:35:33 INFO node.Application: Starting Source source2
17/10/06 14:35:33 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SINK, name: sink2: Successfully registered new MBean.
17/10/06 14:35:33 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: sink2 started
17/10/06 14:35:33 INFO source.AvroSource: Starting Avro source source2: { bindAddress: localhost, port: 10000 }...
17/10/06 14:35:35 ERROR lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:Avro source source2: { bindAddress: localhost, port: 10000 } } - Exception follows.
org.jboss.netty.channel.ChannelException: Failed to bind to: localhost/127.0.0.1:10000
:10000
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
    at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
    at org.apache.flume.source.AvroSource.start(AvroSource.java:242)
    at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.BindException: Address already in use
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:444)
    at sun.nio.ch.Net.bind(Net.java:436)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:372)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:296)
    at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)

我不理解这个错误Caused by: java.net.BindException: Address already in use。 如果我不能使用已经被第一个水槽代理使用的端口,我该如何将它与第二个水槽代理绑定? 或者是单个节点无法完成的事情?

谢谢!

0 个答案:

没有答案