Question

所以我设置了一个来自kafka的风暴喷口和写入HDFS的螺栓。一切正常。我现在想添加一个写入Hbase的新螺栓。出于某种原因，我的应用程序没有拿起hbase配置，我收到以下错误：

            java.lang.IllegalArgumentException: HBase configuration not found using key 'null'
                    at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:58) ~[storm-hbase-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at backtype.storm.daemon.executor$fn__5697$fn__5710.invoke(executor.clj:732) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at backtype.storm.util$async_loop$fn__452.invoke(util.clj:463) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
                    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
            2015-04-16 02:05:44 b.s.d.executor [ERROR]
            java.lang.IllegalArgumentException: HBase configuration not found using key 'null'
                    at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:58) ~[storm-hbase-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at backtype.storm.daemon.executor$fn__5697$fn__5710.invoke(executor.clj:732) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at backtype.storm.util$async_loop$fn__452.invoke(util.clj:463) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
                    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
            2015-04-16 02:05:44 o.a.h.u.NativeCodeLoader [WARN] Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
            2015-04-16 02:05:44 b.s.util [ERROR] Halting process: ("Worker died")
            java.lang.RuntimeException: ("Worker died")
                    at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:322) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na]
                    at backtype.storm.daemon.worker$fn__6109$fn__6110.invoke(worker.clj:495) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at backtype.storm.daemon.executor$mk_executor_data$fn__5530$fn__5531.invoke(executor.clj:245) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at backtype.storm.util$async_loop$fn__452.invoke(util.clj:475) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
                    at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
                    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]

当我查看代码时，它会显示出现错误的以下块：

           public void prepare(Map map, TopologyContext topologyContext, OutputCollector collector) {
                this.collector = collector;
                final Configuration hbConfig = HBaseConfiguration.create();

                Map<String, Object> conf = (Map<String, Object>)map.get(this.configKey);
                if(conf == null) {
                    throw new IllegalArgumentException("HBase configuration not found using key '" + this.configKey + "'");
                }

看起来configKey没有在任何地方设置，所以我尝试将其设置为HBaseBolt方法，如下所示：

    SimpleHBaseMapper mapper = new SimpleHBaseMapper() 
    .withRowKeyField("CustomerId")
    .withColumnFields(new Fields("CustomerId"))
    .withCounterFields(new Fields("Count"))
    .withColumnFamily("cf1");

    HBaseBolt hbase = new HBaseBolt("Customer", mapper).withConfigKey("/etc/hbase/conf/hbase-site.xml");        
    builder.setBolt("HBASE_BOLT", hbase, 1)
    .fieldsGrouping("stormspout", new Fields("CustomerId"));

我似乎没有做任何事情，因为我仍然得到同样的错误.... 有人有什么建议吗？！感觉它只是没有拿起我的hbase-site.xml文件，但我不确定为什么不...

Answer 1

经过大量的工作，我终于开始工作了!!

在拓扑createConfig方法中，添加

    Map<String, String> HBConfig = Maps.newHashMap();
    HBConfig.put("hbase.rootdir","hdfs://<IP Address>:8020/hbase");
    conf.put("HBCONFIG",HBConfig);

实例化HBaseBolt时，请执行此操作 .withConfigKey（＆＃34; HBCONFIG＆＃34）

Answer 2

我实际上最终编写了自己的hbasebolt来实现IRichBolt。然后我对prepare方法进行了覆盖，并构建了以下配置，这似乎解决了我的问题： - ）

            Configuration conf = HBaseConfiguration.create();
            conf.set("hbase.zookeeper.quorum", "sandbox.hortonworks.com"); 
            conf.set("hbase.zookeeper.property.clientPort", "2181");
            conf.set("zookeeper.session.timeout", "1200000");
            conf.set("hbase.zookeeper.property.tickTime", "6000");
            conf.set("zookeeper.znode.parent", "/hbase-unsecure");
            conf.set("hbase.coprocessor.region.classes", "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor");
            conf.set("hbase.coprocessor.master.classes", "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor");  

            admin = new HBaseAdmin(conf);
            table = new HTable(conf, _tableName);

Answer 3

您应该在拓扑配置中为HBase设置设置配置部分名称：

 Config cfg = new Config();
 Map<String, String> HBConfig = Maps.newHashMap();
 HBConfig.put(somekey,somevalue);
 cfg.put("HBCONFIG",HBConfig);
 StormSubmitter.submitTopology(TOPOLOGY_NAME, cfg, builder.createTopology());

然后在您的HBase螺栓配置中将此键设置为螺栓配置键：

  HBaseBolt bolt = new HBaseBolt("table_name", mapper).withConfigKey("HBCONFIG");

未找到Storm Hbase配置

3 个答案: