生成证书颁发机构（确保您的FQDN为`localhost`）

Question

我尝试使用5.1.0-post分支。使用SSL / TLS进行配置时，几乎不会引发任何错误，我还提到了分支，存储库链接和兼容性我已经提到了以下步骤：

＃

   Versions:
a) ElasticSearch => 6.5.4

b) Kafka => 2.1.0

c) Kafka ElasticSearch Connector => 5.1.0-post


Confluent Platform and Apache Kafka Compatibility:
5.0.x   2.0.x
5.1.x   2.1.x
5.2.x   2.2.x

1. `mkdir CONNECTOR_BUILD`

2. `cd CONNECTOR_BUILD`

3. Cloned required packages (Since all 3 below repositories should be at dir level)

    a) `git clone https://github.com/apache/kafka.git --branch 2.1`

    b) `git clone https://github.com/confluentinc/common.git --branch 5.1.0-post`

    c) `git clone https://github.com/confluentinc/kafka-connect-elasticsearch.git --branch 5.1.0-post`

4. Started making package of it.

    a) `cd CONNECTOR_BUILD/kafka`

       i) `gradle` (If gradle not installed, please install it.)

       ii) Run the below command to install jars in local maven repo

       ./gradlew installAll

Build was successful

b) `cd CONNECTOR_BUILD/common`

i) `mvn clean install -DskipTests`

    Build was successful

c) `cd CONNECTOR_BUILD/kafka-connect-elasticsearch`

    i) `mvn clean install -DskipTests`

    Build was successful

5. Untar ES 6.5.4 package.
6. Changed elasticsearch.yml file
7. Untar Kafka 2.1.0 package.
8. Created connector.properties file
9. Created ELK-connector.properties flie 
10.Placed the kafka-connect-elasticsearch-5.1.0.jar (from kafka-connect-elasticsearch) inside kafka_2.11-2.1.0/libs 
11. Also some extra dependencies from kafka-connect-elasticsearch/target/kafka-connect-elasticsearch-5.1.0-package/share/java/kafka-connect-elasticsearch/ inside kafka_2.11-2.1.0/libs 

12. Run the command 

    ./bin/connect-distributed.sh ../connector.properties ../ELK-Connector.properties

I am getting errors when following the below steps:

关注的链接：Kafka Elastic SSL / TLS安全性

cd Analyze / elasticsearch / config
mkdir证书
cd证书
创建的自签名证书。

生成证书颁发机构（确保您的FQDN为`localhost`）

openssl req -new -x509 -keyout cacert.key -out cacert.pem -days 666

生成客户端证书

openssl genrsa -out client1.key 2048

生成证书签名请求

openssl req -new -key client1.key -out client1.csr

与CA签署请求

openssl x509 -req -in client1.csr -CA cacert.pem -CAkey cacert.key -CAcreateserial -out client1.crt -days 1825 -sha256

将连接器密钥打包为JKS

openssl pkcs12 -export -out bundle.p12 -in client1.crt -inkey client1.key


keytool -keystore truststore.jks -import -file cacert.pem -alias cacert


keytool -destkeystore keystore.jks -importkeystore -srckeystore bundle.p12 -srcstoretype PKCS12

配置ElasticSearch配置文件（elasticsearch.yml）

xpack.security.enabled：是 xpack.security.http.ssl.enabled：true xpack.security.http.ssl.client_authentication：必填 xpack.security.http.ssl.key：certs / client1.key xpack.security.http.ssl.certificate：certs / client1.crt xpack.security.http.ssl.certificate_authorities：[“ certs / cacert.pem”]
设置密码 bin/elasticsearch-keystore add

xpack.security.http.ssl.secure_key_passphrase
curl --key config/certs/client1.key --cert config/certs/client1.crt --cacert config/certs/cacert.pem https://10.27.0.114:9200

现在我们可以在下面的输出中看到，ES已启用https。

{
"name" : "mini01",
"cluster_name" : "Gen-E OpsCenter",
"cluster_uuid" : "2qviM7k7SPKp57_8KLDamQ",
"version" :

{ "number" : "6.5.4", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "d2ef93d", "build_date" : "2018-12-17T21:17:40.758843Z", "build_snapshot" : false, "lucene_version" : "7.5.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }
,
"tagline" : "You Know, for Search"
}

配置kafka弹性连接器
现在通过REST调用进行配置设置。由于我们的kafka连接器已启动。

curl -X POST -H "Content-Type: application/json" --data '{"name": "elasticsearch-sink", "config":
{“ connector.class”：“ io.confluent.connect.elasticsearch.ElasticsearchSinkConnector”， “ tasks.max”：“ 4”， “ topics”：“ raw_alarms_topic”，“ topic.index.map”：“ raw_alarms_topic：transport-alarms”，“ key.ignore”：“ true”，“ schema.ignore”：“ true”， “ connection.url”：“ https://mini01:9200”，“ type.name”： “ elasticSearch-sink”，“ elastic.https.ssl.keystore.location”： “ /data/Dhan/KKK/elasticsearch-6.5.4/config/certs/keystore.jks”， “ elastic.https.ssl.keystore.password”：“ qwerty”， “ elastic.https.ssl.key.password”：“ qwerty”， “ elastic.https.ssl.keystore.type”：“ JKS”， “弹性的。https.ssl.truststore.location”： “ /data/Dhan/KKK/elasticsearch-6.5.4/config/certs/truststore.jks”， “ elastic.https.ssl.truststore.password”：“ qwerty”， “ elastic.https.ssl.truststore.type”：“ JKS”， “ elastic.https.ssl.protocol”：“ SSL”}}' http://localhost:8083/connectors

遇到错误：

任务被杀死，并且直到手动重新启动后才能恢复（org.apache.kafka.connect.runtime.WorkerTask：178）[2019-04-15 02：05：24,356]信息正在停止ElasticsearchSinkTask。（io.confluent.connect.elasticsearch.ElasticsearchSinkTask：177） [2019-04-15 02：05：24,415]错误 WorkerSinkTask {id = elasticsearch-sink-1}任务失败了，不可恢复的异常（org.apache.kafka.connect.runtime.WorkerTask：177） org.apache.kafka.connect.errors.ConnectException：无法启动由于连接错误而导致的ElasticsearchSinkTask：at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient。（JestElasticsearchClient.java:145）在 io.confluent.connect.elasticsearch.jest.JestElasticsearchClient。（JestElasticsearchClient.java:112）在 io.confluent.connect.elasticsearch.ElasticsearchSinkTask.start（ElasticsearchSinkTask.java:118）在 io.confluent.connect.elasticsearch.ElasticsearchSinkTask.start（ElasticsearchSinkTask.java:49）在 org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart（WorkerSinkTask.java:302）在 org.apache.kafka.connect.runtime.WorkerSinkTask.execute（WorkerSinkTask.java:191）在 org.apache.kafka.connect.runtime.WorkerTask.doRun（WorkerTask.java:175）在 org.apache.kafka.connect.runtime.WorkerTask.run（WorkerTask.java:219）在 java.util.concurrent.Executors $ RunnableAdapter.call（Executors.java:511）在java.util.concurrent.FutureTask.run（FutureTask.java:266）在 java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1149）在 java.util.concurrent.ThreadPoolExecutor $ Worker.run（ThreadPoolExecutor.java:624）在java.lang.Thread.run（Thread.java:748）造成原因： javax.net.ssl.SSLHandshakeException： sun.security.validator.ValidatorException：PKIX路径构建失败： sun.security.provider.certpath.SunCertPathBuilderException：无法执行在以下位置找到到所需目标的有效认证路径 sun.security.ssl.Alerts.getSSLException（Alerts.java:192）在 sun.security.ssl.SSLSocketImpl.fatal（SSLSocketImpl.java:1946）在 sun.security.ssl.Handshaker.fatalSE（Handshaker.java:316）在 sun.security.ssl.Handshaker.fatalSE（Handshaker.java:310）在 sun.security.ssl.ClientHandshaker.serverCertificate（ClientHandshaker.java:1639）在 sun.security.ssl.ClientHandshaker.processMessage（ClientHandshaker.java:223）在sun.security.ssl.Handshaker.processLoop（Handshaker.java:1037）处 sun.security.ssl.Handshaker.process_record（Handshaker.java:965）在 sun.security.ssl.SSLSocketImpl.readRecord（SSLSocketImpl.java:1064）位于 sun.security.ssl.SSLSocketImpl.performInitialHandshake（SSLSocketImpl.java:1367）在 sun.security.ssl.SSLSocketImpl.startHandshake（SSLSocketImpl.java:1395）在 sun.security.ssl.SSLSocketImpl.startHandshake（SSLSocketImpl.java:1379）在 org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket（SSLConnectionSocketFactory.java:394）在 org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket（SSLConnectionSocketFactory.java:353）在 org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect（DefaultHttpClientConnectionOperator.java:141）在 org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect（PoolingHttpClientConnectionManager.java:353）在 org.apache.http.impl.execchain.MainClientExec。EstablishmentRoute（MainClientExec.java:380）在 org.apache.http.impl.execchain.MainClientExec.execute（MainClientExec.java:236）在 org.apache.http.impl.execchain.ProtocolExec.execute（ProtocolExec.java:184）在 org.apache.http.impl.execchain.RetryExec.execute（RetryExec.java:88）在 org.apache.http.impl.execchain.RedirectExec.execute（RedirectExec.java:110）在 org.apache.http.impl.client.InternalHttpClient.doExecute（InternalHttpClient.java:184）在 org.apache.http.impl.client.CloseableHttpClient.execute（CloseableHttpClient.java:82）在 org.apache.http.impl.client.CloseableHttpClient.execute（CloseableHttpClient.java:107）在 io.searchbox.client.http.JestHttpClient.executeRequest（JestHttpClient.java:118）在 io.searchbox.client.http.JestHttpClient.execute（JestHttpClient.java:57）在 io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.getServerVersion（JestElasticsearchClient.java:166）在 io.confluent.connect.elasticsearch.jest.JestElasticsearchClient。（JestElasticsearchClient.java:143） ... 12更多原因：sun.security.validator.ValidatorException： PKIX路径构建失败： sun.security.provider.certpath.SunCertPathBuilderException：无法执行在以下位置找到到所需目标的有效认证路径 sun.security.validator.PKIXValidator.doBuild（PKIXValidator.java:397）在 sun.security.validator.PKIXValidator.engineValidate（PKIXValidator.java:302）在sun.security.validator.Validator.validate（Validator.java:262）处 sun.security.ssl.X509TrustManagerImpl.validate（X509TrustManagerImpl.java:324）在 sun.security.ssl.X509TrustManagerImpl.checkTrusted（X509TrustManagerImpl.java:229）在 sun.security.ssl.X509TrustManagerImpl.checkServerTrusted（X509TrustManagerImpl.java:124）在 sun.security.ssl.ClientHandshaker.serverCertificate（ClientHandshaker.java:1621） ... 35更多原因： sun.security.provider.certpath.SunCertPathBuilderException：无法执行在以下位置找到到所需目标的有效认证路径 sun.security.provider.certpath.SunCertPathBuilder.build（SunCertPathBuilder.java:141）在 sun.security.provider.certpath.SunCertPathBuilder.engineBuild（SunCertPathBuilder.java:126）在java.security.cert.CertPathBuilder.build（CertPathBuilder.java:280）在 sun.security.validator.PKIXValidator.doBuild（PKIXValidator.java:392） ... 41更多[2019-04-15 02：05：24,417]错误 WorkerSinkTask {id = elasticsearch-sink-1}任务正在被杀死，并且将在手动重启之前无法恢复（org.apache.kafka.connect.runtime.WorkerTask：178）

使用elasticsearch和kafka-elastic-connector设置SSL / TLS配置时出错

生成证书颁发机构（确保您的FQDN为`localhost`）

生成客户端证书

生成证书签名请求

与CA签署请求

将连接器密钥打包为JKS

0 个答案:

使用elasticsearch和kafka-elastic-connector设置SSL / TLS配置时出错

生成证书颁发机构（确保您的FQDN为localhost）

生成客户端证书

生成证书签名请求

与CA签署请求

将连接器密钥打包为JKS

0 个答案:

生成证书颁发机构（确保您的FQDN为`localhost`）