Question

我试图以独立模式启动弹性搜索接收器连接器。

我的quickstart-elasticsearch.properties

name=elasticsearch-sink
connector.class=io.confluent.connect.elasticsearch.ElasticsearchSinkConnector
tasks.max=3
topics=fsp-audit
key.ignore=true
connection.url=https://***.amazonaws.com
type.name=kafka-connect
errors.tolerance = all
errors.deadletterqueue.topic.name = fsp-dlq-audit-event

我的connect-standalone.properties

bootstrap.servers=***9092
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
schema.enabled=false
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000
plugin.path=/usr/local/confluent/share/java

我像下面那样启动我的连接器

/usr/local/confluent/bin/connect-standalone /usr/local/confluent/etc/kafka/connect-standalone.properties /usr/local/confluent/etc/kafka-connect-elasticsearch/quickstart-elasticsearch.properties

但是最后我开始低于错误

[2020-01-31 06:47:40,994] INFO Using multi thread/connection supporting pooling connection manager (io.searchbox.client.JestClientFactory:224)
[2020-01-31 06:47:41,047] INFO Using default GSON instance (io.searchbox.client.JestClientFactory:68)
[2020-01-31 06:47:41,047] INFO Node Discovery disabled... (io.searchbox.client.JestClientFactory:85)
[2020-01-31 06:47:41,048] INFO Idle connection reaping disabled... (io.searchbox.client.JestClientFactory:97)
[2020-01-31 06:47:41,223] ERROR WorkerSinkTask{id=elasticsearch-sink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
java.lang.NullPointerException
        at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.getServerVersion(JestElasticsearchClient.java:231)
        at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.<init>(JestElasticsearchClient.java:142)
        at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.<init>(JestElasticsearchClient.java:133)
        at io.confluent.connect.elasticsearch.ElasticsearchSinkTask.start(ElasticsearchSinkTask.java:122)
        at io.confluent.connect.elasticsearch.ElasticsearchSinkTask.start(ElasticsearchSinkTask.java:51)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:301)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:189)
        at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
[2020-01-31 06:47:41,224] ERROR WorkerSinkTask{id=elasticsearch-sink-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:180)
[2020-01-31 06:47:41,224] INFO Stopping ElasticsearchSinkTask (io.confluent.connect.elasticsearch.ElasticsearchSinkTask:190)

这是我的示例消息

{
  "ID": "avro2-9749-0e710000fd04",
  "VERSION": "1",
  "ACTION_TYPE": "NEW_CASE",
  "EVENT_TYPE": "WORLDCHECK",
  "CLIENT_ID": "fgh-5d1e-17a2-9749-0e4d00",
  "DETAILS": "<?xml version=\"1.0\" <gender>MALE</gender></caseCreatedPayload>",
  "OBJECT_TYPE": "CASE",
  "UTC_DATE_TIME": "1578469623000",
  "POINT_IN_TIME_PRECISION": "TIME",
  "TIME_ZONE": "UTC",
  "TIMELINE_PRECISION": "ON",
  "GROUP_ID": "0a348753-5d1e-17a2-9749-0e4d0000146d",
  "OBJECT_DISPLAY_NAME": "NULL",
  "OBJECT_ID": "0a348753-5d1e-17af-9749-0e7100006ccf",
  "USER_DISPLAY_NAME": "USER_FIRST_6cb4c322-cd3d-4809-97d3-07d2d96f10ed",
  "USER_ID": "USER_LAST_7e99cad9-dc1c-4770-ac4f-33c4897ce404",
  "PARENT_EVENT_ID": "0a348752-5d17-138e-9749-0e6a00000c7f",
  "NOTES": "null",
  "SUMMARY": "sumary",
  "AUDIT_EVENT_TO_UTC_DT": "1578469621000",
  "AUDIT_EVENT_TO_DATE_PITP": "null",
  "AUDIT_EVENT_TO_DATE_TZ": "null",
  "AUDIT_EVENT_TO_DATE_TP": "null"
}

我正在AWS和EC2上运行我的所有设置

Answer 1

您的堆栈跟踪显示JestElasticsearchClient.getServerVersion，这意味着您的ES连接URL不正确。例如，您输入https时未提供任何凭据，或者未列出端口

即使您只有一个连接实例，也建议使用分布式模式

在独立模式下启动Kafka连接时出现java.lang.NullPointerException

1 个答案: