Question

我使用下面的示例代码将一些测试数据从Confluent Kafka加载到Hbase，但不知何故我看不到数据来自Hbase。我尝试了一切，但根本没用。任何人都可以帮忙吗？

如何复制问题：

下载示例代码：

https://github.com/mravi/kafka-connect-hbase

编译示例代码

mvn -DskipTests = true package
修改ConnectDistributed服务器上的hbase-sink.properties

vim ./etc/kafka-connect-hbase/hbase-sink.properties

name=kafka-cdc-hbase
connector.class=io.svectors.hbase.sink.HBaseSinkConnector
tasks.max=1
topics=test
zookeeper.quorum=wdsjnl012.test.com:2181
event.parser.class=io.svectors.hbase.parser.AvroEventParser

hbase.test.rowkey.columns=id

hbase.test.rowkey.delimiter=,

hbase.test.family=name

按以下命令启动ConnectDistributed：

./ bin / confluent start connect
创建Hbase表：

HBase的（主）：007：0＆GT;创建'test'，'name'

HBase的（主）：007：0＆GT;描述'测试' 表测试已启用
测试
栏目家庭描述
{NAME =＆gt; 'name'，BLOOMFILTER =＆gt; 'ROW'，VERSIONS =＆gt; '1'，IN_MEMORY =＆gt; 'false'，KEEP_DELETED_CELLS =＆gt; 'FALSE'，DATA_BLOCK_EN CODING =＆gt; '无'，TTL =＆gt; 'FOREVER'，COMPRESSION =＆gt; 'NONE'，MIN_VERSIONS =＆gt; '0'，BLOCKCACHE =＆gt; 'true'，BLOCKSIZE =＆gt; '65536' ，REPLICATION_SCOPE =＆gt; '0'}
生成数据：

./ bin / kafka-avro-console-producer --broker-list wdsjnl010：9092 - topic test --property value.schema ='{“type”：“record”，“name”：“record”，“fields”： [{“name”：“id”，“type”：“int”}，{“name”：“name”，“type”：“string”}]}'

{“id”：1，“name”：“foo”}

顺便说一句，我仔细检查hbase connect是否已加载日志文件，我认为已经加载了hbase connect plugin：

[2018-05-18 19:48:32,714] INFO Loading plugin from: 
/usr/local/confluent/share/java/kafka-connect-hbase 
(org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:194)
[2018-05-18 19:48:35,447] INFO Registered loader: 
PluginClassLoader{pluginLocation=file:/usr/local/confluent/share/java/kafka- 
connect-hbase/} 
(org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:217)
[2018-05-18 19:48:35,447] INFO Added plugin 
'io.svectors.hbase.sink.HBaseSinkConnector' 
(org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:146)
[2018-05-18 19:48:35,447] INFO Loading plugin from: 
/usr/local/confluent/share/java/confluent-common 
(org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:194)
[2018-05-18 19:48:35,516] INFO Registered loaded:
PluginClassLoader{pluginLocation=file:/usr/local/confluent/share/java/confluent-common/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:217)
    [2018-05-18 19:48:35,518] INFO Loading plugin from: /usr/local/confluent/share/java/kafka-connect-s3 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:194)
[2018-05-18 19:48:38,396] INFO Registered loader: PluginClassLoader{pluginLocation=file:/usr/local/confluent/share/java/kafka-connect-s3/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:217)
[2018-05-18 19:48:38,396] INFO Added plugin 'io.confluent.connect.s3.S3SinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:146)
[2018-05-18 19:48:40,673] INFO Registered loader: sun.misc.Launcher$AppClassLoader@764c12b6 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:217)
[2018-05-18 19:48:40,675] INFO Added aliases 'ElasticsearchSinkConnector' and 'ElasticsearchSink' to plugin 'io.confluent.connect.elasticsearch.ElasticsearchSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,675] INFO Added aliases 'HdfsSinkConnector' and 'HdfsSink' to plugin 'io.confluent.connect.hdfs.HdfsSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,676] INFO Added aliases 'JdbcSinkConnector' and 'JdbcSink' to plugin 'io.confluent.connect.jdbc.JdbcSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,676] INFO Added aliases 'JdbcSourceConnector' and 'JdbcSource' to plugin 'io.confluent.connect.jdbc.JdbcSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,676] INFO Added aliases 'S3SinkConnector' and 'S3Sink' to plugin 'io.confluent.connect.s3.S3SinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,676] INFO Added aliases 'HBaseSinkConnector' and 'HBaseSink' to plugin 'io.svectors.hbase.sink.HBaseSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,676] INFO Added aliases 'FileStreamSinkConnector' and 'FileStreamSink' to plugin 'org.apache.kafka.connect.file.FileStreamSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,677] INFO Added aliases 'FileStreamSourceConnector' and 'FileStreamSource' to plugin 'org.apache.kafka.connect.file.FileStreamSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,677] INFO Added aliases 'MockConnector' and 'Mock' to plugin 'org.apache.kafka.connect.tools.MockConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,677] INFO Added aliases 'MockSinkConnector' and 'MockSink' to plugin 'org.apache.kafka.connect.tools.MockSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,677] INFO Added aliases 'MockSourceConnector' and 'MockSource' to plugin 'org.apache.kafka.connect.tools.MockSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,677] INFO Added aliases 'VerifiableSinkConnector' and 'VerifiableSink' to plugin 'org.apache.kafka.connect.tools.VerifiableSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,677] INFO Added aliases 'VerifiableSourceConnector' and 'VerifiableSource' to plugin 'org.apache.kafka.connect.tools.VerifiableSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,678] INFO Added aliases 'AvroConverter' and 'Avro' to plugin 'io.confluent.connect.avro.AvroConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,678] INFO Added aliases 'ByteArrayConverter' and 'ByteArray' to plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,678] INFO Added aliases 'JsonConverter' and 'Json' to plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,678] INFO Added aliases 'StringConverter' and 'String' to plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,678] INFO Added aliases 'ByteArrayConverter' and 'ByteArray' to plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,678] INFO Added aliases 'JsonConverter' and 'Json' to plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,678] INFO Added alias 'SimpleHeaderConverter' to plugin 'org.apache.kafka.connect.storage.SimpleHeaderConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:348)
[2018-05-18 19:48:40,678] INFO Added aliases 'StringConverter' and 'String' to plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:351)
[2018-05-18 19:48:40,680] INFO Added alias 'RegexRouter' to plugin 'org.apache.kafka.connect.transforms.RegexRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:348)
[2018-05-18 19:48:40,680] INFO Added alias 'TimestampRouter' to plugin 'org.apache.kafka.connect.transforms.TimestampRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:348)
[2018-05-18 19:48:40,680] INFO Added alias 'ValueToKey' to plugin 'org.apache.kafka.connect.transforms.ValueToKey' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:348)
[2018-05-18 19:48:40,700] INFO DistributedConfig values:

无法将Confluent Kafka主题中的avro数据用于Hbase

0 个答案: