Question

我正在尝试使用Confluent Kafka Sink连接器连接到Azure Data Explorer。 https://github.com/Azure/kafka-sink-azure-kusto 将记录添加到源（MsSql）时，它无法将数据推送到接收器。

Aws Ec2实例具有用于Kafka-connect / Zookeeper / control-center等的提供的融合docker容器。用于kafka接收器连接器的jar放置在plugin文件夹中。 Docker-compose确保此连接器通过卷可用于kafka-connect docker容器。使用控制中心UI连接MsSql源连接器和Kusto接收器连接器。将记录添加到MsSql时，它将引发以下异常-

[2019-06-06 00:19:02,140] INFO Refreshing Ingestion Resources (com.microsoft.azure.kusto.ingest.ResourceManager)
[2019-06-06 00:19:02,140] ERROR ingestFromFile: Error ingesting local file: /var/tmp/sinklog/kafka_kafkatest1_0_0.gz (com.microsoft.azure.kusto.ingest.IngestClientImpl)
java.lang.IllegalArgumentException: username is null or empty
    at com.microsoft.aad.adal4j.AuthenticationContext.acquireToken(AuthenticationContext.java:205)
    at com.microsoft.azure.kusto.data.AadAuthenticationHelper.acquireAadUserAccessToken(AadAuthenticationHelper.java:55)
    at com.microsoft.azure.kusto.data.AadAuthenticationHelper.acquireAccessToken(AadAuthenticationHelper.java:43)
    at com.microsoft.azure.kusto.data.KustoClient.execute(KustoClient.java:34)
    at com.microsoft.azure.kusto.data.KustoClient.execute(KustoClient.java:30)
    at com.microsoft.azure.kusto.data.KustoClient.execute(KustoClient.java:20)
    at com.microsoft.azure.kusto.ingest.ResourceManager.refreshIngestionResources(ResourceManager.java:127)
    at com.microsoft.azure.kusto.ingest.ResourceManager.getIngestionResource(ResourceManager.java:104)
    at com.microsoft.azure.kusto.ingest.IngestClientImpl.ingestFromFile(IngestClientImpl.java:109)
    at com.microsoft.azure.kusto.kafka.connect.sink.TopicPartitionWriter.handleRollFile(TopicPartitionWriter.java:43)
    at com.microsoft.azure.kusto.kafka.connect.sink.GZIPFileWriter.finishFile(GZIPFileWriter.java:100)
    at com.microsoft.azure.kusto.kafka.connect.sink.GZIPFileWriter.rotate(GZIPFileWriter.java:93)
    at com.microsoft.azure.kusto.kafka.connect.sink.GZIPFileWriter.write(GZIPFileWriter.java:57)
    at com.microsoft.azure.kusto.kafka.connect.sink.TopicPartitionWriter.writeRecord(TopicPartitionWriter.java:79)
    at com.microsoft.azure.kusto.kafka.connect.sink.KustoSinkTask.put(KustoSinkTask.java:203)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:565)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:323)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:226)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:194)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

接收器配置如下：

{
  "name": "KustoSink1",
  "connector.class": "com.microsoft.azure.kusto.kafka.connect.sink.KustoSinkConnector",
  "tasks.max": "1",
  "key.converter": "org.apache.kafka.connect.storage.StringConverter",
  "value.converter": "org.apache.kafka.connect.storage.StringConverter",
  "config.action.reload": "restart",
  "errors.retry.timeout": "0",
  "errors.log.enable": "true",
  "errors.log.include.messages": "true",
  "topics": "kafkatest1",
  "errors.deadletterqueue.topic.name": "",
  "errors.deadletterqueue.topic.replication.factor": "3",
  "errors.deadletterqueue.context.headers.enable": "false",
  "kusto.url": "https://ingest-xxxx.kusto.windows.net",
  "kusto.db": "kafkaadx01",
  "kusto.table": "kafkatest",
  "kusto.tables.topics_mapping": "[{'topic': 'kafkatest1','db': 'kafkaadx01', 'table': 'kafkatest','format': 'json', 'mapping':'TestMapping'}]",
  "kusto.auth.username": "xxx",
  "kusto.auth.password": "xxxx",
  "kusto.auth.appid": "",
  "kusto.auth.appkey": "",
  "kusto.auth.authority": "XXXX",
  "kusto.sink.tempdir": "/var/tmp/sinklog",
  "kusto.sink.flush_size": "1024",
  "kusto.sink.flush_interval_ms": "1000"
}

我使用相同的凭据从相同的环境执行了一个python程序，并且它起作用了（这是从python程序读取的操作）。因此凭据似乎很好，也没有防火墙问题。

请帮助我解决这个问题。

Azure Kafka Sink连接器无法连接到Azure Data Explorer

0 个答案: