Question

我在一台EC2机器（M1）上运行了我的日志和logstash，因此我使用此配置读取了放置在本地计算机上的日志：

input { 
    file{
        path => "/path/to/logs/in/M1"
        start_position => "beginning"
    }
}

现在，我们在不同的EC2机器（M2）上运行弹性搜索，我需要使用logstash将日志从M1传输到M2中的elasticsearch。我使用了以下输出配置：

output {
    stdout { codec => rubydebug }

    elasticsearch {
            hosts => "http://<M2 ip address>:9200"
        index => "logstash-%{+YYYY.MM.dd}"
    }
}

当我运行配置文件时，出现以下错误：

04:18:57.640 [[main]>worker0] WARN  logstash.outputs.elasticsearch - UNEXPECTED POOL ERROR {:e=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections>}
04:18:57.646 [[main]>worker0] ERROR logstash.outputs.elasticsearch - Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>2}
04:18:59.682 [[main]>worker0] WARN  logstash.outputs.elasticsearch - UNEXPECTED POOL ERROR {:e=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections>}
04:18:59.686 [[main]>worker0] ERROR logstash.outputs.elasticsearch - Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>4}
04:19:01.109 [Ruby-0-Thread-17: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-5.4.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:188] WARN  logstash.outputs.elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x1d08c988 URL:http://10.60.40.120:9200>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://10.60.40.120:9200][Manticore::ConnectTimeout] connect timed out"}
04:19:02.111 [Ruby-0-Thread-17: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-5.4.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:188] INFO  logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:url=>#<URI::HTTP:0x55444fcf URL:http://10.60.40.120:9200>, :healthcheck_path=>"/"}

我是logstash的新手。任何帮助表示赞赏。

更新：

所以我查看了forumns，我得到了一个解决方案，告诉我使用命令更新logstash输出：

sudo /usr/share/logstash/bin/logstash-plugin update logstash-output-elasticsearch

我还更新了logstash配置文件以包含用户名和密码：

output {
    stdout { codec => rubydebug }

    elasticsearch {
            hosts => ["<M2 ip address>"]
        user => 'username'
        password => 'changeme'
        index => "logstash-%{+YYYY.MM.dd}"
        manage_template => false
    }
}

现在我收到了一个不同的错误。请求帮助：

09:16:21.305 [[main]>worker0] WARN  logstash.outputs.elasticsearch - Could not index event to Elasticsearch. {:status=>404, :action=>["index", {:_id=>nil, :_index=>"logstash-2017.04.17", :_type=>"Messagelog", :_routing=>nil}, 2017-04-17T10:06:11.348Z ip-10-60-40-201 No valid licenses found for COLL], :response=>{"index"=>{"_index"=>"logstash-2017.04.17", "_type"=>"Messagelog", "_id"=>nil, "status"=>404, "error"=>{"type"=>"index_not_found_exception", "reason"=>"no such index and [action.auto_create_index] ([.security,.monitoring*,.watches,.triggered_watches,.watcher-history*]) doesn't match", "index_uuid"=>"_na_", "index"=>"logstash-2017.04.17"}}}}

感谢。

Answer 1

您似乎已禁用在elasticsearch上自动创建索引。默认情况下，elasticsearch支持自动创建索引。删除

action.auto_create_index: -b*,+a*,-*

你的elasticsearch.yml中的

（无论模式如何）你会很好。

此外，如果您想接受以l开头的自动创建索引，请使用模式+ l *。那就是添加

action.auto_create_index: +l*

阅读this了解更多信息。

使用logstash将一台计算机中的日志加载到另一台计算机中的elasticsearch设置中

1 个答案: