Question

我在OSX Mavericks上使用Elasticsearch-Exporter对弹性搜索执行转储：

node /usr/bin/node_modules/elasticsearch-exporter/exporter.js -j ${esIndexName} -f esbackup

我有一个运行两个节点的应用程序，它与应用程序节点一起总共增加了三个节点。 elasticsearch命令创建的节点是主节点。当我对我的索引运行export命令时，我会在成功加载几秒后得到它：

2014-05-07T14:31:38.325-0700 [elasticsearch[Rancor][[es][1]: Lucene Merge Thread #0]] [WARN] merge.scheduler [][] - [Rancor] [es][1] failed to merge
 815 java.io.FileNotFoundException: /private/var/data/core/elasticsearch_me/nodes/0/indices/es/1/index/_f_es090_0.tip (Too many open files)

我尝试过以下方法：

launchctl limit 10000

sudo launchctl limit 40000 65000

elasticsearch soft nofile 32000

elasticsearch hard nofile 32000

将-XX:-MaxFDLimit添加到我的应用程序的jvm参数

这些都没有解决我的问题。偶尔负载将完成没有错误，但大多数时候我遇到错误。有没有人对我的问题有什么想法/提示？

编辑：

$ launchctl limit cpu unlimited unlimited filesize unlimited unlimited data unlimited unlimited stack 8388608 67104768 core 0 unlimited rss unlimited unlimited memlock unlimited unlimited maxproc 709 1064 maxfiles 10000 10240

$ sudo launchctl limit Password: cpu unlimited unlimited filesize unlimited unlimited data unlimited unlimited stack 8388608 67104768 core 0 unlimited rss unlimited unlimited memlock unlimited unlimited maxproc 709 1064 maxfiles 40000 65000

Answer 1

好的 - 如果你在一台Mac上运行多个elasticsearch节点加上node.js应用程序我肯定会确保你的打开文件数量达到了ES推荐的限制：

file descriptor

Make sure to increase the number of open files descriptors on the machine (or for the user running elasticsearch). Setting it to 32k or even 64k is recommended.

In order to test how many open files the process can open, start it with -Des.max-open-files set to true. This will print the number of open files the process can open on startup.

Alternatively, you can retrieve the max_file_descriptors for each node using the Nodes Info API, with:

curl localhost:9200/_nodes/process?pretty

您需要确保为运行ES的用户完成此操作，而不仅仅是root用户（当然，除非您以root身份运行它）。

为此，请遵循以下指示：（http://elasticsearch-users.115913.n3.nabble.com/quot-Too-many-open-files-quot-error-on-Mac-OSX-td4034733.html）。假设你想要32k和64k作为限制：

In /etc/launchd.conf put:

limit maxfiles 32000 64000


Make sure in your ~/.bashrc file you are not setting the ulimit with something like "ulimit -n 1024".  

Open a new terminal, and run:

launchctl limit maxfiles
ulimit -a

进行这些更改后，请不要忘记重新启动。然后当您在此命令行参数中启动elasticsearch传递时：

elasticsearch -XX:-MaxFDLimit

在我的Mac上执行上述步骤后，我从Elasticsearch获得以下响应：

curl http://localhost:9200/_nodes/process?pretty

{
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "XXXXXXXXXXXXXXXXXXXXXXX" : {
      "name" : "Marrina Smallwood",
      "transport_address" : "inet[XX.XX.XX.XX:9300]",
      "host" : "MacBook-Pro-Retina.local",
      "ip" : "XX.XX.XX.XX",
      "version" : "1.1.1",
      "build" : "f1585f0",
      "http_address" : "inet[/XX.XX.XX.XX:9200]",
      "process" : {
        "refresh_interval" : 1000,
        "id" : 538,
        "max_file_descriptors" : 32768,
        "mlockall" : false
      }
    }
  }
}

＆＃34;太多打开的文件＆＃34;同时将数据转储到elasticsearch

1 个答案: