Question

我在使用flume

提取twitter数据时使用了此命令

[cloudera @ localhost bin] $ ./flume-ng agent --conf ./conf/-f ../conf/flume.conf -Dflume.root.logger = DEBUG，console -n TwitterAgent

2015-07-14 05:42:00 [INFO ] Configuration provider starting
2015-07-14 05:42:00 [INFO ] Reloading configuration file:../conf/flume.conf
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Added sinks: HDFS Agent: TwitterAgent
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Processing:HDFS
2015-07-14 05:42:00 [INFO ] Post-validation flume configuration contains configuration for agents: [TwitterAgent]
2015-07-14 05:42:00 [INFO ] Creating channels
2015-07-14 05:42:00 [INFO ] Creating instance of channel MemChannel type memory
2015-07-14 05:42:00 [INFO ] Created channel MemChannel
2015-07-14 05:42:00 [INFO ] Creating instance of source Twitter, type org.apache.flume.source.twitter.TwitterSource

正在处理这个，然后是错误：

2015-07-14 05:42:01 [错误] Unhand-led错误 java.lang.NoSuchMethodError： twitter4j.TwitterStream.addListener（Ltwitter4j / StatusListener;）V 在org.apache.flume.source.twitter.TwitterSource.configure（TwitterSource.java:119）在org.apache.flume.conf.Configurables.configure（Configurables.java:41） at org.apache.flume.node.AbstractConfigurationProvider.loadSources（AbstractConfigurationProvider.java:331） at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration（AbstractConfigurationProvider.java:102） at org.apache.flume.node.PollingPropertiesFileConfigurationProvider $ FileWatcherRunnable.run（PollingPropertiesFileConfigurationProvider.java:140） at java.util.concurrent.Executors $ RunnableAdapter.call（Executors.java:441） at java.util.concurrent.FutureTask $ Sync.innerRunAndReset（FutureTask.java:317） at java.util.concurrent.FutureTask.runAndReset（FutureTask.java:150） at java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.access $ 101（ScheduledThreadPoolExecutor.java:98） at java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.runPeriodic（ScheduledThreadPoolExecutor.java:180） at java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run（ScheduledThreadPoolExecutor.java:204） at java.util.concurrent.ThreadPoolExecutor $ Worker.runTask（ThreadPoolExecutor.java:886） at java.util.concurrent.ThreadPool Executor $ Worker.run（ThreadPoolExecutor.java:908） at java.lang.Thread.run（Thread.java:662)2015-07-14 05:44:13 [INFO]停止生命周期主管10 2015-07-14 05:44:13 [INFO] 配置提供程序停止

Answer 1

java.lang.NoSuchMethodError＆lt; - 这表示您使用的是错误版本的twitter4j。

当前1.6.0版本的flume的twitter-source是针对3.0.3构建的：

[INFO] +- org.twitter4j:twitter4j-core:jar:3.0.3:compile
[INFO] +- org.twitter4j:twitter4j-media-support:jar:3.0.3:compile
[INFO] \- org.twitter4j:twitter4j-stream:jar:3.0.3:compile

只需将twitter4j lib替换为那些，然后就可以了。

hadoop flume提取twitter数据

1 个答案: