Question

我为水槽创建了一个自定义源，并将jar文件复制到以下位置：

 mkdir -p /usr/lib/flume-ng/plugins.d/MyFlumeSource/lib/MyFlumeSource.jar

 chown -R flume:flume /var/lib/flume-ng/

同样在/etc/flume-ng/conf/flume-env.sh

FLUME_CLASSPATH="/usr/lib/flume-ng/plugins.d/MyFlumeSource/lib/MyFlumeSource.jar"

将Flume配置文件更新为

 # Name the components on this agent 
tail1.sources = seq-source  
tail1.channels = mem-channel
tail1.sinks = hdfs-sink

# Describe/configure Source
tail1.sources.seq-source.type = org.custom.flume.source.MySource

# Describe the sink
tail1.sinks.hdfs-sink.type = hdfs
tail1.sinks.hdfs-sink.hdfs.path = /user/flume
tail1.sinks.hdfs-sink.hdfs.filePrefix = log
tail1.sinks.hdfs-sink.hdfs.rollInterval = 0
tail1.sinks.hdfs-sink.hdfs.rollCount = 10000
tail1.sinks.hdfs-sink.hdfs.fileType = DataStream

# Use a channel which buffers events in file
tail1.channels.mem-channel.type = memory
tail1.channels.mem-channel.capacity = 1000
tail1.channels.mem-channel.transactionCapacity = 100

# Bind the source and sink to the channel
tail1.sources.seq-source.channels = mem-channel
tail1.sinks.hdfs-sink.channel = mem-channel

尝试以

运行水槽剂

flume-ng agent  --conf /var/lib/flume-ng/plugins.d/MyFlumeSource/lib/MyFlumeSource.jar --conf-file /etc/flume-ng/conf/flume-conf.properties --name tail1

flume-ng agent --conf-file /etc/flume-ng/conf/flume-conf.properties --name tail1

在这两种情况下，我都收到以下错误：

 ERROR node.PollingPropertiesFileConfigurationProvider: Failed to load configuration data. Exception follows.
org.apache.flume.FlumeException: Unable to create source: seq-source, type: org.custom.flume.source.MySource, class: org.custom.flume.source.MySource
        at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:48)
        at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322)
        at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97)
        at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InstantiationException
        at sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at java.lang.Class.newInstance(Class.java:379)
        at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:44)
        ... 10 more

如果有人知道，请帮助我。

Answer 1

您在/var/lib/flume-ng下安装了该插件，但是您尝试从/etc/flume-ng和--conf运行该插件。

此外，必须说<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>Login Form</title> </head> <body> <form action="Login_Servlet_Test" method="POST"> Username <input type="text" name="uname"/><br> Password <input type="text" name="paswd"/><br> <input type="Submit" value="Submit"/> </form> </body> </html>选项应该指向整个配置文件夹。

Answer 2

你在寻找水槽拦截器吗？您可以将类添加为处理消息的拦截器。如果是，您可以通过2个简单步骤完成。 1）将jar文件添加到/ lib 2）在flume-conf.properties中添加配置，提及您的Builder类名称。

水槽中的定制来源

2 个答案: