Question

我已经在 EKS 集群上部署了 Logstash 和 elasticsearch pod。当我检查 logstash pod 的日志时，它显示无法访问的 elasticserach 服务器。尽管我的 elasticsearch 已启动并正在运行。请找到以下 yaml 文件并记录错误。

configMap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: "logstash-configmap-development"
  namespace: "development"
  labels:
    app: "logstash-development"
data:
  logstash.conf: |-
    input {
         http {
         
      }
    }
    
    filter {
        json {
            source => "message"
        }
    }
    output {
      elasticsearch {
            hosts => ["https://my-server.com/elasticsearch-development/"]
            index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
        }
        
        stdout {
            codec => rubydebug
      }
    }

deployment.yaml

---
  apiVersion: "apps/v1"
  kind: "Deployment"
  metadata: 
    name: "logstash-development"
    namespace: "development"
  spec: 
    selector: 
      matchLabels: 
        app: "logstash-development"
    replicas: 1
    strategy: 
      type: "RollingUpdate"
      rollingUpdate: 
        maxSurge: 1
        maxUnavailable: 1
    minReadySeconds: 5
    template: 
      metadata: 
        labels: 
          app: "logstash-development"
      spec: 
        containers: 
          - 
            name: "logstash-development"
            image: "logstash:7.10.2"
            imagePullPolicy: "Always"
            env: 
              - 
                name: "XPACK_MONITORING_ELASTICSEARCH_HOSTS"
                value: "https://my-server.com/elasticsearch-development/"
                
              -
                name: "XPACK_MONITORING_ELASTICSEARCH_URL"
                value: "https://my-server.com/elasticsearch-development/"
                
              -
              
                name: "SERVER_BASEPATH"
                value: "logstash-development"
                
                
                
            securityContext:
                privileged: true
                
            ports: 
              - 
                containerPort: 8080
                protocol: TCP
                
            volumeMounts:
                -
                  name: "logstash-conf-volume"
                  mountPath: "/usr/share/logstash/pipeline/"
                  
        volumes:
            -
              name: "logstash-conf-volume"
              configMap:
                 name: "logstash-configmap-development"
                 items:
                    - key: "logstash.conf"
                      path: "logstash.conf"
        imagePullSecrets: 
          - 
            name: "logstash"

service.yaml

---
  apiVersion: "v1"
  kind: "Service"
  metadata: 
    name: "logstash-development"
    namespace: "development"
    labels: 
      app: "logstash-development"
  spec: 
    ports: 
      - 
        port: 55770
        targetPort: 8080
    selector: 
      app: "logstash-development"

Logstash pod 日志错误

[2021-06-09T08:22:38,708][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://my-server.com/elasticsearch-development/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://my-server.com/elasticsearch-development/][Manticore::ConnectTimeout] connect timed out"}

注意：- Elasticsearch 已启动并正在运行。当我点击 logstash url 时，它显示状态正常。

我已经检查过 elasticsearch cluster-ip，他们的 logstash 能够与 Elasticsearch 连接，但是当我提供入口路径 url 时，它无法连接到 elasticsearch。

同样从日志中，我注意到它为 elasticsearch 使用了不正确的 url。

我的 elasticsearch 网址是这样的：- https://my-server.com/elasticserach

但 logstash 正在寻找 https://my-server.com:9200/elasticsearch

使用此 url (https://my-server.com:9200/elasticsearch) 无法访问 elasticsearch，因为它导致连接超时。

有人能说出为什么需要 (https://my-server.com:9200/elasticsearch) 而不是 (https://my-server.com/elasticsearch)

Answer 1

我现在可以将logstash与elasticsearch连接起来，如果您使用的是带有dns名称的elasticsearch，默认情况下logstash会将elasticsearch的端口设为9200，因此在我的情况下，它会将elasticsearch url设为https://my-server.com:9200/elasticsearch-development/ .但是使用该 url，elasticsearch 无法访问，只能通过 (https://myserver.com/elasticsearch-development/) 访问。因此，我需要在我的 elasticsearch url 中添加 https 端口，即 443，通过该端口，logstash 将能够连接到 elasticserach (https://my-server.com:443/elasticsearch-development/)

长话短说：-

在环境变量 XPACK_MONITORING_ELASTICSEARCH_HOSTS 和 XPACK_MONITORING_ELASTICSEARCH_URL 下的 deployment.yaml 文件中，给定值为 https://my-server.com:443/elasticsearch-development/

在 logstash.conf 文件中给出了相同的值。

Logstash 无法连接到部署在 Kubernetes 集群上的 Elasticsearch

1 个答案: