indexing - Solr索引文档很晚

在某些情况下，当我将文档提供给Solr进行索引时，该文档似乎没有被索引，或者仅被索引得很晚。 SolrCloud 7.1.0会发生这种情况。

我的案子：
-在08：59：48.157我添加了一个文档A
-在08：59：48.264，我添加了一个文档B
-在09：00：19.467，我执行查询并找到 only 文档A。

这是在自动测试中发生的，但不会经常复制。在大约90％的情况下，它都可以正常工作（同时显示A和B），而在另外10％的情况下，我没有两个文件。

我已将 autoCommit 设置为15秒（openSearcher=false），将 autoSoftCommit 设置为1秒（我知道此设置应尽可能高）稍后再增加）。

我假设Solr会使用自动（软）提交进行日志记录，并且确实确实看到了DirectUpdateHandler2的“ commitScheduler” 线程进行的日志记录，但是运行很少。在同时添加文档A和B之后， commitScheduler 在日志中的首次出现是在9:00:25，这是在添加文档之后将近40秒。

从索引中删除和对象时，似乎出现了相同的问题。有时它只是没有发生，或者至少很晚。我在日志中看到“删除”，并且50秒后触发的查询仍然会产生已删除的对象。

当我将成功运行的日志与失败运行的日志进行比较时，我看不到任何区别。 SolrCloud日志（运行失败）：

2018-09-14 08:59:48.144 INFO  (zkCallback-3-thread-4-processing-n:localhost:5100_solr) [   ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 34 ms
2018-09-14 08:59:48.151 INFO  (Thread-80) [   ] o.a.s.s.IndexSchema [Cloud_shard1_replica_n1] Schema name=base-schema
2018-09-14 08:59:48.156 INFO  (Thread-80) [   ] o.a.s.s.IndexSchema Loaded schema base-schema/1.6 with uniqueid field id

2018-09-14 08:59:48.157 INFO  (qtp947679291-17) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [Cloud_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5f8ecb57-2135-4c26-a9b3-6808531badd0 (1611572799756828672)]} 0 9

2018-09-14 08:59:48.160 INFO  (Thread-80) [   ] o.a.s.c.CoreContainer Reloading SolrCore 'Cloud_shard1_replica_n1' using configuration from collection Cloud
2018-09-14 08:59:48.160 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.c.SolrCore [[Cloud_shard1_replica_n1] ] Opening new SolrCore at [/var/app/current/solr/work/Cloud_shard1_replica_n1], dataDir=[/var/app/current/solr/work/Cloud_shard1_replica_n1/data/]
2018-09-14 08:59:48.181 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.h.a.SystemInfoHandler Resolving canonical hostname for local host prevented due to 'solr.dns.prevent.reverse.lookup' sysprop
2018-09-14 08:59:48.186 WARN  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
2018-09-14 08:59:48.187 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
2018-09-14 08:59:48.187 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 1000ms; 
2018-09-14 08:59:48.202 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3844c2e[Cloud_shard1_replica_n1] main]
2018-09-14 08:59:48.203 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/tenant
2018-09-14 08:59:48.203 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/tenant
2018-09-14 08:59:48.203 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/tenant/managed-schema
2018-09-14 08:59:48.204 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 66 is already the latest
2018-09-14 08:59:48.204 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
2018-09-14 08:59:48.204 INFO  (searcherExecutor-540-thread-1-processing-n:localhost:5100_solr x:Cloud_shard1_replica_n1 s:shard1 c:Cloud r:core_node2) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.c.SolrCore [Cloud_shard1_replica_n1] Registered new searcher Searcher@3844c2e[Cloud_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.0):C1870) Uninverting(_1(7.2.0):C4) Uninverting(_2(7.2.0):C1)))}
2018-09-14 08:59:48.227 INFO  (qtp947679291-15) [   ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
2018-09-14 08:59:48.228 INFO  (qtp947679291-15) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
2018-09-14 08:59:48.237 INFO  (qtp947679291-15) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.c.S.Request [Cloud_shard1_replica_n1]  webapp=/solr path=/schema/fields params={wt=javabin&version=2} status=0 QTime=0
2018-09-14 08:59:48.252 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.

2018-09-14 08:59:48.264 INFO  (qtp947679291-15) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [Cloud_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8e209dd4-03ef-4397-8c6a-b947270af684 (1611572799877414912)]} 0 2

2018-09-14 08:59:48.269 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@73887417[Cloud_shard1_replica_n1] main]
2018-09-14 08:59:48.270 INFO  (searcherExecutor-540-thread-1-processing-n:localhost:5100_solr x:Cloud_shard1_replica_n1 s:shard1 c:Cloud r:core_node2) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.c.SolrCore [Cloud_shard1_replica_n1] Registered new searcher Searcher@73887417[Cloud_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.0):C1870) Uninverting(_1(7.2.0):C4) Uninverting(_2(7.2.0):C1)))}
2018-09-14 08:59:48.270 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.c.SolrCore [Cloud_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2460d222
2018-09-14 08:59:48.271 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.Cloud.shard1.replica_n1, tag=610325026
2018-09-14 08:59:48.271 INFO  (Thread-80) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.Cloud.shard1.leader, tag=610325026

2018-09-14 09:00:19.467 INFO  (qtp947679291-18) [c:Cloud s:shard1 r:core_node2 x:Cloud_shard1_replica_n1] o.a.s.c.S.Request [Cloud_shard1_replica_n1]  webapp=/solr path=/query    params={q=*&df=_text_&qt=/query&_stateVer_=Cloud:4&fl=id,_displayName&start=0&sort=_displayName+asc&fq=(MY-FQ)&rows=1000&wt=javabin&version=2} hits=1 status=0 QTime=1

似乎只有 first 测试偶尔会失败。在启动SolrCloud之后立即执行此测试。我还没有看到以后的测试失败。

任何建议将不胜感激。

Solr索引文档很晚

1 个答案: