Question

嗨我有数据弹性80 M数据有52 Go和我用这个查询在文本中搜索

$json_doc = '{
                 "query": {
                    "query_string": {
                        "query": "*' . $sujet . '*",
                        "lenient": true
                    }
                    },
     "size": 10000000,
     "from": 0,
     "fields": [
     "association_key","text"
                    ]
                }';

当我搜索基础中的单词看起来少于20K时，我有我的数据但是当他超过20 K我有这个错误

[2015-11-14 20:03:04,663][DEBUG][action.search.type       ] [Pixx] [myindex3][2], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][2]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
    at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
    at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
    at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
    at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:56)
    at org.apache.lucene.search.HitQueue.<init>(HitQueue.java:64)
    at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:275)
    at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:37)
    at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:135)
    at org.apache.lucene.search.TopScoreDocCollector$OutOfOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:133)
    at org.apache.lucene.search.TopScoreDocCollector.create(TopScoreDocCollector.java:262)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:490)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
    ... 8 more
[2015-11-14 20:03:11,764][DEBUG][action.search.type       ] [Pixx] [myindex3][0], node[x9gOi0WuQviXeE0eM6MsAg], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@3989d22b] lastShard [true]
org.elasticsearch.search.query.QueryPhaseExecutionException: [myindex3][0]: query[filtered(_all:*startup _all:maroc*)->cache(_type:tweecoms3)],from[0],size[10000000]: Query Failed [Failed to execute main query]
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
    at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:286)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:297)
    at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
    at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
    at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.lucene.util.FixedBitSet.<init>(FixedBitSet.java:187)
    at org.elasticsearch.common.lucene.docset.DocIdSets.toCacheable(DocIdSets.java:81)
    at org.elasticsearch.index.cache.filter.weighted.WeightedFilterCache$FilterCacheFilterWrapper.getDocIdSet(WeightedFilterCache.java:177)
    at org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:46)
    at org.apache.lucene.search.FilteredQuery$1.bulkScorer(FilteredQuery.java:144)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:618)
    at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:191)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:281)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:269)
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:157)
    ... 8 more

我如何解决这个问题我专用服务器只包含一个英特尔至强W3520 c 4/8 t + 2.66 GHz内存32 GB ECC Debian

Answer 1

好像你没有足够的空间容纳Java的堆。由于您似乎拥有更多的硬件，因此您可以尝试增加它： Increase heap size in Java

Answer 2

弹性搜索无法在未配置的情况下使用所有内存。

您需要配置和调整Elastic Search和Debian的内存设置。

您需要配置的主要内容之一是：

ES_HEAP_SIZE ：内存弹性搜索应保留
ES_JAVA_OPTS ：精确的JVM设置，包括内存和GC调整（如果需要）

在此处查看更多信息： https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html

查询失败[无法执行主查询] OutOfMemoryError：Java堆空间

2 个答案: