我正在尝试使用Cloudera VM设置一个简单的两层水槽代理。 代码如下:
# First-tier agent
agent1.sources = source1
agent1.sinks = sink1
agent1.channels = channel1
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channel = channel1
agent1.sources.source1.type = spooldir
agent1.sources.source1.spoolDir = /tmp/spooldir
agent1.sinks.sink1.type = avro
agent1.sinks.sink1.hostname = localhost
agent1.sinks.sink1.port = 10000
agent1.channels.channel1.type = file
agent1.channels.channel1.checkpointDir = /tmp/flume/channel1/check
agent1.channels.channel1.dataDirs = /tmp/flume/channel1/data
# Second-tier agent
agent2.sources = source2
agent2.sinks = sink2
agent2.channels = channel2
agent2.sources.source2.channels = channel2
agent2.sinks.sink2.channel = channel2
agent2.sources.source2.type = avro
agent2.sources.source2.bind = localhost
agent2.sources.source2.port = 10000
agent2.sinks.sink2.type = hdfs
agent2.sinks.sink2.hdfs.path = /tmp/flume
agent2.sinks.sink2.hdfs.filePrefix = events
agent2.sinks.sink2.hdfs.fileSuffix = .log
agent2.sinks.sink2.hdfs.fileType = DataStream
agent2.channels.channel2.type = file
agent2.channels.channel2.checkpointDir = /tmp/flume/channel2/check
agent2.channels.channel2.dataDirs = /tmp/flume/channel2/data
第一个代理看起来不错,但是一旦我运行第二个代理,它的源就无法连接到第一个代理的接收器。 它给出了以下错误:
17/10/06 14:35:33 INFO node.Application: Starting Sink sink2
17/10/06 14:35:33 INFO node.Application: Starting Source source2
17/10/06 14:35:33 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SINK, name: sink2: Successfully registered new MBean.
17/10/06 14:35:33 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: sink2 started
17/10/06 14:35:33 INFO source.AvroSource: Starting Avro source source2: { bindAddress: localhost, port: 10000 }...
17/10/06 14:35:35 ERROR lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:Avro source source2: { bindAddress: localhost, port: 10000 } } - Exception follows.
org.jboss.netty.channel.ChannelException: Failed to bind to: localhost/127.0.0.1:10000
:10000
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
at org.apache.flume.source.AvroSource.start(AvroSource.java:242)
at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:372)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:296)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
我不理解这个错误Caused by: java.net.BindException: Address already in use
。
如果我不能使用已经被第一个水槽代理使用的端口,我该如何将它与第二个水槽代理绑定?
或者是单个节点无法完成的事情?
谢谢!