我是Hadoop(Cloudera)的新手,我只是想通过水槽传输推文,我想将它们保存在Hbase中。但是,我遇到了例外,目前不知道如何解决此问题。
我有一个SplittingSerializer-0.0.1-SNAPSHOT.jar,它存储在Flume-ng / lib目录中。
在我的flume-env.sh文件中,指定了以下目录:
FLUME_CLASSPATH="/opt/cloudera/parcels/lib/flume-ng/lib/SplittingSerializer-0.0.1-SNAPSHOT.jar"
在twitter.conf文件中,我输入了以下参数:
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = hbaseSink
TwitterAgent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sources.Twitter.consumerKey = xxx
TwitterAgent.sources.Twitter.consumerSecret = xxx
TwitterAgent.sources.Twitter.accessToken = xxx
TwitterAgent.sources.Twitter.accessTokenSecret = xxx
TwitterAgent.sources.Twitter.keywords = IoT
TwitterAgent.sinks.hbaseSink.type=org.apache.flume.sink.hbase.AsyncHBaseSink
TwitterAgent.sinks.hbaseSink.channel=MemChannel
TwitterAgent.sinks.hbaseSink.table=hbtweet
TwitterAgent.sinks.hbaseSink.columnFamily=tweet
TwitterAgent.sinks.hbaseSink.serializer=org.apache.flume.sink.hbase.SplittingSerializer
TwitterAgent.sinks.hbaseSink.serializer.columns=tweet:nothing
TwitterAgent.channels.MemChannel.type = memory
TwitterAgent.channels.MemChannel.capacity = 10000
TwitterAgent.channels.MemChannel.transactionCapacity = 100
然后我执行以下命令:
flume-ng agent -n TwitterAgent -c conf -f /opt/cloudera/parcels/lib/flume-ng/conf/twitter.conf
在HBase中,我创建了一个表:
create 'hbtweet', 'tweet'
我收到以下错误消息:
18/09/10 22:21:56 INFO zookeeper.ClientCnxn: Opening socket connection to server mrhel/xx:xx. Will not attempt to authenticate using SASL (unknown error)
18/09/10 22:21:56 INFO async.HBaseClient: Need to find the -ROOT- region
18/09/10 22:21:56 INFO hbase.AsyncHBaseSink: waiting on callback
18/09/10 22:21:56 INFO zookeeper.ClientCnxn: Socket connection established to mrhel/xx:xx, initiating session
18/09/10 22:21:56 INFO zookeeper.ClientCnxn: Session establishment complete on server mrhel/xx:xx, sessionid = xx, negotiated timeout = 5000
18/09/10 22:21:56 ERROR zookeeper.ClientCnxn: Caught unexpected throwable
java.lang.NoSuchMethodError: org.hbase.async.generated.ZooKeeperPB$MetaRegionServer.makeExtensionsImmutable()V
at org.hbase.async.generated.ZooKeeperPB$MetaRegionServer.<init>(ZooKeeperPB.java:128)
at org.hbase.async.generated.ZooKeeperPB$MetaRegionServer.<init>(ZooKeeperPB.java:62)
at org.hbase.async.generated.ZooKeeperPB$MetaRegionServer$1.parsePartialFrom(ZooKeeperPB.java:137)
at org.hbase.async.generated.ZooKeeperPB$MetaRegionServer$1.parsePartialFrom(ZooKeeperPB.java:132)
at org.hbase.async.generated.ZooKeeperPB$MetaRegionServer$Builder.mergeFrom(ZooKeeperPB.java:409)
at org.hbase.async.generated.ZooKeeperPB$MetaRegionServer$Builder.mergeFrom(ZooKeeperPB.java:324)
at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:108)
at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:162)
at org.hbase.async.HBaseClient$ZKClient$ZKCallback.handleMetaZnode(HBaseClient.java:3753)
at org.hbase.async.HBaseClient$ZKClient$ZKCallback.processResult(HBaseClient.java:3621)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:558)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
有人知道我可以解决这个问题吗?