Kibana刷新

时间:2016-10-10 07:59:35

标签: elasticsearch elastic-stack

我正面临着Elasticsearch的一个问题,我看到它在我的服务器中占用了176%的CPU,所以我想知道这是这个版本中的错误还是堆栈本身。它不是在中间发生,但在两个小时之后,我看到CPU使用率上的这些峰值,有时会因负载而导致服务器无响应。

是否还有人面临这个问题?

我的java版本是

$~# java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

我更新了Elasticsearch中的配置,以便消耗更少的CPU和内存。现在我的elasticsearch.yml配置就像这样

indices.fieldata.cache.size: "30%"
indices.breaker.fielddata.limit: "80%"
indices.breaker.request.limit: "50%"
indices.breaker.total.limit: "90%"
bootstrap.mlockall: true
index.merge.scheduler.max_thread_count: 1
indices.store.throttle.type: none

然而,我仍然看到Elasticsearch的CPU使用高峰,当kibana刷新仪表板时,我已经识别出它们的出现,我可以看到> 100%(107,165 ......)的峰值。奇怪的是内存使用情况,净I / O和块I / O似乎保持稳定,而CPU峰值

对此有何建议或想法?

提前谢谢你,

问候!

我用顶部录制的日志来搜索CPU峰值

  

root @ g:〜#cat cpu-log.txt | grep colord

     

6.9 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.8 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.8 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.8 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.7 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.7 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.6 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.6 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.6 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.6 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.6 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.5 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.5 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.5 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

6.5 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

7.3 389 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

83.5 5864 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch - cp /usr/share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

57.5 5864 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX :CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp / usr /share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

110 6949 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java   -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF- 8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

     

86.6 8100 colord / usr / lib / jvm / java-8-openjdk-amd64 / jre / bin / java -Xms256m -Xmx1g -Djava.awt.headless = true -XX:+ UseParNewGC -XX:+ UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction = 75 -XX:+ UseCMSInitiatingOccupancyOnly -XX:+ HeapDumpOnOutOfMemoryError -XX:+ DisableExplicitGC -Dfile.encoding = UTF-8 -Djna.nosys = true -Des.path.home = / usr / share / elasticsearch - cp /usr/share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/*   org.elasticsearch.bootstrap.Elasticsearch start   -Des.network.host = 0.0.0.0

1 个答案:

答案 0 :(得分:0)

调试CPU使用率并不是一项简单的任务,需要更多信息来确定导致这些峰值的原因,例如集群大小,数据量,分片和副本数量,索引映射和还有您可视化的仪表板的属性。

通常,在Elasticsearch上查询和聚合时,应该看到CPU使用量增加,增加的程度取决于我上面列出的信息。

作为一个起点,我建议您按照此production guide进行Elasticsearch,因为我发现您在某些默认设置下运行,而且您可能还想先使用Chrome网络工具重新加载仪表板可以查看发送到Elasticsearch的网络请求,运行对/_msearch请求的所有响应,并检查每个查询的took值,这样可以让您了解ES可能出现的大量查询挣扎着。