持续获取以下警告消息。不确定应该做什么。看到一些要求增加文件描述符数量的相关帖子。
如何做同样的事情？
即使我现在增加，我是否会在添加新指数时遇到同样的问题。 （目前使用大约400个索引，6个分片和1个副本）。指数的数量往往会增长更多。
[03:58:24,165][WARN ][cluster.action.shard ] [node1] received shard failed for [index9][2], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index9][2] failed recovery]; nested: EngineCreationFailureException[[index9][2] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index9/2/index/segments_1 (Too many open files)]; ]]
[03:58:24,166][WARN ][cluster.action.shard ] [node1] received shard failed for [index15][0], node[node_hash2], [P], s[INITIALIZING], reason [Failed to create shard, message [IndexShardCreationException[[index15][0] failed to create shard]; nested: IOException[directory '/data/elasticsearch/whatever/nodes/0/indices/index15/0/index' exists and is a directory, but cannot be listed: list() returned null]; ]]
[03:58:24,195][WARN ][cluster.action.shard ] [node1] received shard failed for [index16][3], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index16][3] failed recovery]; nested: EngineCreationFailureException[[index16][3] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index16/3/index/segments_1 (Too many open files)]; ]]
[03:58:24,196][WARN ][cluster.action.shard ] [node1] received shard failed for [index17][0], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index17][0] failed recovery]; nested: EngineCreationFailureException[[index17][0] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index17/0/index/segments_1 (Too many open files)]; ]]
[03:58:24,198][WARN ][cluster.action.shard ] [node1] received shard failed for [index21][4], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index21][4] failed recovery]; nested: EngineCreationFailureException[[index21][4] failed to create engine]; nested: LockReleaseFailedException[Cannot forcefully unlock a NativeFSLock which is held by another indexer component: /data/elasticsearch/whatever/nodes/0/indices/index21/4/index/write.lock]; ]]
节点api的输出
curl -XGET 'http://localhost:9200/_nodes?os=true&process=true&pretty=true'
{
"ok" : true,
"cluster_name" : "whatever",
"nodes" : {
"node_hash1" : {
"name" : "node1",
"transport_address" : "transportip1",
"hostname" : "myhostip1",
"version" : "0.20.4",
"http_address" : "httpip1",
"attributes" : {
"data" : "false",
"master" : "true"
},
"os" : {
"refresh_interval" : 1000,
"available_processors" : 8,
"cpu" : {
"vendor" : "Intel",
"model" : "Xeon",
"mhz" : 2133,
"total_cores" : 8,
"total_sockets" : 8,
"cores_per_socket" : 16,
"cache_size" : "4kb",
"cache_size_in_bytes" : 4096
},
"mem" : {
"total" : "7gb",
"total_in_bytes" : 7516336128
},
"swap" : {
"total" : "30gb",
"total_in_bytes" : 32218378240
}
},
"process" : {
"refresh_interval" : 1000,
"id" : 26188,
"max_file_descriptors" : 16384
}
},
"node_hash2" : {
"name" : "node2",
"transport_address" : "transportip2",
"hostname" : "myhostip2",
"version" : "0.20.4",
"attributes" : {
"master" : "false"
},
"os" : {
"refresh_interval" : 1000,
"available_processors" : 4,
"cpu" : {
"vendor" : "Intel",
"model" : "Xeon",
"mhz" : 2400,
"total_cores" : 4,
"total_sockets" : 4,
"cores_per_socket" : 32,
"cache_size" : "20kb",
"cache_size_in_bytes" : 20480
},
"mem" : {
"total" : "34.1gb",
"total_in_bytes" : 36700303360
},
"swap" : {
"total" : "0b",
"total_in_bytes" : 0
}
},
"process" : {
"refresh_interval" : 1000,
"id" : 24883,
"max_file_descriptors" : 16384
}
},
"node_hash3" : {
"name" : "node3",
"transport_address" : "transportip3",
"hostname" : "myhostip3",
"version" : "0.20.4",
"attributes" : {
"master" : "false"
},
"os" : {
"refresh_interval" : 1000,
"available_processors" : 4,
"cpu" : {
"vendor" : "Intel",
"model" : "Xeon",
"mhz" : 2666,
"total_cores" : 4,
"total_sockets" : 4,
"cores_per_socket" : 16,
"cache_size" : "8kb",
"cache_size_in_bytes" : 8192
},
"mem" : {
"total" : "34.1gb",
"total_in_bytes" : 36700303360
},
"swap" : {
"total" : "0b",
"total_in_bytes" : 0
}
},
"process" : {
"refresh_interval" : 1000,
"id" : 25328,
"max_file_descriptors" : 16384
}
}
}
答案 0 :(得分：13)
如何增加允许打开文件的最大数量取决于您的Linux发行版。以下是ubuntu和centos的一些说明：
http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ http://pro.benjaminste.in/post/318453669/increase-the-number-of-file-descriptors-on-centos-and
elasticsearch文档建议将最大文件限制设置为32k或64k。既然你已达到16k并且已经达到了极限，我可能会把它设置得更高;像128k这样的东西。请参阅：http://www.elasticsearch.org/guide/reference/setup/installation/
在增加打开文件的数量并重新启动elasticsearch之后，您需要通过重新运行您提到的curl命令来验证它是否有效：
curl -XGET 'http://localhost:9200/_nodes?os=true&process=true&pretty=true'
当您添加更多索引（以及更多文档）时，您还会看到elasticsearch跟踪增加的文件数。如果您注意到所有指标和文档的性能下降，您可以尝试向群集添加新节点：http://www.elasticsearch.org/guide/reference/setup/configuration/ - 因为您已经有一个分片，复制配置，这应该是一个相对无痛的过程。
答案 1 :(得分：3)
停止ElasticSearch。如果你从命令开始 （bin / elasticsearch）然后请具体这个设置堆 开始。例如，我使用16GB的盒子，所以我的命令是
一个。 bin / elasticsearch -Xmx8g -Xms8g
湾转到config（elasticsearch / config / elasticsearch.yml）并确保
bootstrap.mlockall：true
℃。将 ulimits -Hn 和 ulimits -Sn 增加到超过200000
如果您是作为服务开始的，请执行以下操作
一个。导出ES_HEAP_SIZE = 10g 湾转到config（/etc/elasticsearch/elasticsearch.yml）并确保
bootstrap.mlockall：true C。将 ulimits -Hn 和 ulimits -Sn 增加到超过200000
确保您输入的大小不超过堆的50％，无论您是将其作为服务启动还是从命令行启动
答案 2 :(得分：1)
请注意，如果elasticsearch是/etc/security/limits.conf服务，则通过systemd更改ulimit无效。
要increase the elasticsearch limit under systemd创建具有以下内容的文件/etc/systemd/system/elasticsearch.service.d/override.conf：
[服务]
LimitNOFILE =无穷大
然后运行systemctl daemon-reload && systemctl restart elasticsearch。