Question

我们正在使用 6 节点运行elasticsearch-1.5.1群集。最近几天我在群集中遇到 java.lang.OutOfMemoryError PermGen space 问题节点和相同的节点会下降。我正在重新启动特定节点以实现生效。

我们试图通过给群集带来沉重的负担来解决这个问题，但不幸的是无法重现。但是我们如何在生产中一次又一次地得到同样的问题。

这里有一些yml文件配置

index.recovery.initial_shards: 1
index.query.bool.max_clause_count: 8192
index.mapping.attachment.indexed_chars: 500000
index.merge.scheduler.max_thread_count: 1
cluster.routing.allocation.node_concurrent_recoveries: 15
indices.recovery.max_bytes_per_sec: 50mb
indices.recovery.concurrent_streams: 5

内存配置

ES_HEAP_SIZE=10g
ES_JAVA_OPTS="-server -Des.max-open-files=true"
MAX_OPEN_FILES=65535
MAX_MAP_COUNT=262144

使用以下配置更新问题

我怀疑merge.policy.max_merged_segment与此问题有关。我的群集中有22个索引。索引的merge.policy.max_merged_segment在下面给出

7个索引有20gb
3个指数有10gb
12个指数有5gb

使用流程信息进行更新

esuser xxxxx 1 Oct03？ 1-02：20：40 / usr / java / default / bin / java -Xms10g -Xmx10g -Djava.awt.headless = true -XX：+ UseParNewGC -XX：+ UseConcMarkSweepGC -XX：CMSInitiatingOccupancyFraction = 75 -XX：+ UseCMSInitiatingOccupancyOnly -XX：+ HeapDumpOnOutOfMemoryError -XX：+ DisableExplicitGC -Dfile.encoding = UTF-8 -server -Des.max-open-files = true -Delasticsearch -Des.pidfile = / var / es / elasticsearch.pid -Des.path.home = / usr / es / elasticsearch -cp：/usr/es/elasticsearch/lib/elasticsearch-1.5.1.jar:/ USR / ES / elasticsearch / LIB / ：/ USR / ES / elasticsearch / LIB / SIGAR / -Des.default.path.home = / usr / es / elasticsearch -Des.default.path.logs = / es / es_logs -Des.default.path.data = / es / es_data -Des.default.path.work = / es / es_work -Des.default.path.conf = / etc / elasticsearch org.elasticsearch.bootstrap.Elasticsearch

我在搜索时从弹性搜索群集获取的堆栈跟踪下方。但事件虽然索引时间也是我得到同样的问题。根据我的观察，一些搜索/索引操作增加了PermGen，如果即将到来的操作试图使用PermGen空间问题来了。

[2015-10-03 06:45:05,262][WARN ][transport.netty          ] [es_f2_01] Message not fully read (response) for [19353573] handler org.elasticsearch.search.action.SearchServiceTransportAction$6@21a25e37, error [true], resetting
[2015-10-03 06:45:05,262][DEBUG][action.search.type       ] [es_f2_01] [product_index][4], node[GoUqK7csTpezN5_xoNWbeg], [R], s[INITIALIZING]: Failed to execute [org.elasticsearch.action.search.SearchRequest@5c2fe4c4] lastShard [true]
org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream
Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream
    at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:176)
    at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:128)
    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.OutOfMemoryError: PermGen space

能帮助我解决这个问题。感谢

Answer 1

最好的解决方案是使用＆＃34; Java 8＆＃34; JVM。

虽然您可以修改Java 7 JVM正在使用的堆的数量（通过设置-XX：MaxPermSize = ...如果您使用的是Oracle JVM），如果您只是将JVM升级到版本8，那么您不需要甚至需要调整permgen的大小。

这是因为在JVM 8中，permgen大小以非分区方式共享堆，这意味着当用完堆时，只会耗尽permgen空间。

在运行elasticsearch集群时遇到Perm Gen空间问题

1 个答案: