我正在尝试将本地logstash连接到Amazon Elasticsearch Service。出现以下错误。
[2018-06-28T08:12:23,165][ERROR][logstash.inputs.metrics ] X-Pack is installed on Logstash but not on Elasticsearch. Please install X-Pack on Elasticsearch to use the monitoring feature. Other features may be available.
[2018-06-28T08:12:23,186][DEBUG][logstash.agent ] Starting puma
[2018-06-28T08:12:23,195][DEBUG][logstash.agent ] Trying to start WebServer {:port=>9600}
[2018-06-28T08:12:23,238][DEBUG][logstash.api.service ] [api-service] start
[2018-06-28T08:12:23,284][DEBUG][logstash.pipeline ] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x85f0f01@/<localpath>/logstash-6.3.0/logstash-core/lib/logstash/pipeline_action/create.rb:48 sleep>"}
[2018-06-28T08:12:23,387][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-06-28T08:12:07,737][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://vpc-<hidden-vpc-elastic-url>:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://vpc-<hidden-vpc-elastic-url>:9200/][Manticore::ConnectTimeout] connect timed out"}
下面是我的本地logstash配置。我不确定s3配置的前缀是否是file / file_name的路径。
input{
s3 {
bucket => "<bucket_name>"
access_key_id => "<access_key_hidden>"
secret_access_key => "<secret_access_key_hidden>"
region => "us-west-1"
prefix => "staging/Staging.json"
}
}
filter {
json {
source => "message"
target => "staging_collection"
remove_field => [ "message" ]
}
}
output {
elasticsearch {
hosts => ["<vpc_name>"]
ssl => true
index => "staging"
document_type => "stagingorders"
document_id => "%{[staging_collection][entityId]}"
}
}
我能够在AWS中卷曲到Elasticsearch服务的VPC网址。我的logstash.yml设置了这些属性。应该设置这些属性以便连接到vpc elasticsearch服务并推送数据。
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: ["https://vpc-<vpc_url>"]
xpack.monitoring.elasticsearch.sniffing: false