我们正在测试环境中运行具有3个节点的solr 7集群。 我们有一个自定义请求处理程序/响应编写器，用于在solr中添加一个端点以支持我们的业务逻辑。
硬件配置：32 GB RAM，8核。
当服务器上的负载适度增加（例如每秒1-3个请求）时，我们看到很多线程正在等待锁。
我们所有方面字段都为docValues = true。 我们还将使用大约20-25个方面的字段和大约30-40个方面的查询来进行繁琐的方面分析。
我的solr索引中的文档数约为：550万
qtp1597462040-382 thread obtained org.apache.solr.uninverting.FieldCache$CreationPlaceholder's lock & did not release it. Due to that 125 threads are BLOCKED as shown in the below graph. If threads are BLOCKED for prolonged period, application will become unresponsive. Examine 'qtp1597462040-382' stacktrace to see why lock is not released.
The stacktrace of the thread that is blocking all the other similar threads is as follows:
qtp1597462040-382 - threadId:382 - state:RUNNABLE
stackTrace:
java.lang.Thread.State: RUNNABLE
at org.apache.lucene.codecs.lucene50.ForUtil.readBlock(ForUtil.java:203)
at org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockDocsEnum.refillDocs(Lucene50PostingsReader.java:357)
at org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockDocsEnum.nextDoc(Lucene50PostingsReader.java:382)
at org.apache.solr.uninverting.DocTermOrds.uninvert(DocTermOrds.java:371)
at org.apache.solr.uninverting.DocTermOrds.<init>(DocTermOrds.java:222)
at org.apache.solr.uninverting.DocTermOrds.<init>(DocTermOrds.java:213)
at org.apache.solr.uninverting.DocTermOrds.<init>(DocTermOrds.java:206)
at org.apache.solr.uninverting.FieldCacheImpl$DocTermOrdsCache.createValue(FieldCacheImpl.java:1249)
at org.apache.solr.uninverting.FieldCacheImpl$Cache.get(FieldCacheImpl.java:199)
- locked <3dad1e71> (a org.apache.solr.uninverting.FieldCache$CreationPlaceholder)
at org.apache.solr.uninverting.FieldCacheImpl.getDocTermOrds(FieldCacheImpl.java:1236)
at org.apache.solr.uninverting.UninvertingReader.getSortedSetDocValues(UninvertingReader.java:369)
at org.apache.lucene.index.FilterLeafReader.getSortedSetDocValues(FilterLeafReader.java:390)
at org.apache.lucene.index.DocValues.getSortedSet(DocValues.java:429)
at org.apache.lucene.queries.function.valuesource.SortedSetFieldSource.getValues(SortedSetFieldSource.java:59)
at org.apache.lucene.queries.function.valuesource.SimpleBoolFunction.getValues(SimpleBoolFunction.java:48)
at org.apache.lucene.queries.function.valuesource.SimpleBoolFunction.getValues(SimpleBoolFunction.java:35)
at org.apache.lucene.queries.function.valuesource.SimpleBoolFunction.getValues(SimpleBoolFunction.java:48)
at org.apache.lucene.queries.function.valuesource.SimpleBoolFunction.getValues(SimpleBoolFunction.java:35)
at org.apache.lucene.queries.function.valuesource.MultiBoolFunction.getValues(MultiBoolFunction.java:49)
at org.apache.lucene.queries.function.valuesource.MultiBoolFunction.getValues(MultiBoolFunction.java:33)
at org.apache.lucene.queries.function.valuesource.IfFunction.getValues(IfFunction.java:47)
at org.apache.lucene.queries.function.valuesource.MultiFloatFunction.getValues(MultiFloatFunction.java:76)
at org.apache.lucene.queries.function.valuesource.MultiFloatFunction.getValues(MultiFloatFunction.java:76)
at org.apache.lucene.queries.function.valuesource.DualFloatFunction.getValues(DualFloatFunction.java:56)
at org.apache.lucene.queries.function.ValueSource$ValueSourceComparator.doSetNextReader(ValueSource.java:429)
at org.apache.lucene.search.SimpleFieldComparator.getLeafComparator(SimpleFieldComparator.java:36)
at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:180)
at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector.getLeafCollector(TopFieldCollector.java:122)
at org.apache.lucene.search.MultiCollector.getLeafCollector(MultiCollector.java:121)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:651)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462)
at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:215)
at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1698)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1415)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1435)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:375)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at com.my.match.index.solr.MySearchRequestHandler.handleRequestBody(MySearchRequestHandler.java:472)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:530)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEnd
Point$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
除此之外，在以下堆栈跟踪中我还看到其他一些线程很忙
at org.apache.solr.request.DocValuesFacets.accumMultiSeg(DocValuesFacets.java:366)
at org.apache.solr.request.DocValuesFacets.accumMulti(DocValuesFacets.java:326)
at org.apache.solr.request.DocValuesFacets.getCounts(DocValuesFacets.java:152)
at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:595)
at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426)
at org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$0(SimpleFacets.java:834)
at org.apache.solr.request.SimpleFacets$$Lambda$365/532253544.call(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.request.SimpleFacets$3.execute(SimpleFacets.java:773)
at com.my.match.index.solr.MySearchRequestHandler.handleRequestBody(MySearchRequestHandler.java:472)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
有两个字段是'text_general'并经过分析并且仍在构面中使用。那会是个问题吗？我知道在这些领域进行刻面可能会很困难。
我们还能分享什么更多信息？