Lucene索引损坏?

时间:2018-07-31 18:12:44

标签: java lucene hibernate-search

我们正在使用休眠搜索(5.6.5.Final)+ lucene(5.5.5),并且在查询索引时注意到某些服务器上出现间歇性问题。

我不确定要尝试缩小问题范围,但我不确定100%需要提供什么信息,但是我们在日志中看到了堆栈跟踪,如下所示:

 java.lang.ArrayIndexOutOfBoundsException
 at org.apache.lucene.store.ByteArrayDataInput.readVInt(ByteArrayDataInput.java:105)
 at org.apache.lucene.codecs.blocktree.IntersectTermsEnumFrame.nextNonLeaf(IntersectTermsEnumFrame.java:254)
 at org.apache.lucene.codecs.blocktree.IntersectTermsEnumFrame.next(IntersectTermsEnumFrame.java:239)
 at org.apache.lucene.codecs.blocktree.IntersectTermsEnum.popPushNext(IntersectTermsEnum.java:345) 
 at org.apache.lucene.codecs.blocktree.IntersectTermsEnum._next(IntersectTermsEnum.java:713)
 at org.apache.lucene.codecs.blocktree.IntersectTermsEnum.next(IntersectTermsEnum.java:501)
 at org.apache.lucene.index.FilteredTermsEnum.next(FilteredTermsEnum.java:224)
 at org.apache.lucene.search.FuzzyTermsEnum.next(FuzzyTermsEnum.java:240)
 at org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:67)
 at org.apache.lucene.search.TopTermsRewrite.rewrite(TopTermsRewrite.java:67)
 at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:331)
 at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:278)
 at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:278)
 at org.hibernate.search.query.engine.impl.LazyQueryState.rewrittenQuery(LazyQueryState.java:192)
 at org.hibernate.search.query.engine.impl.LazyQueryState.search(LazyQueryState.java:103)
 at org.hibernate.search.query.engine.impl.QueryHits.updateTopDocs(QueryHits.java:241)
 at org.hibernate.search.query.engine.impl.QueryHits.<init>(QueryHits.java:136)
 at org.hibernate.search.query.engine.impl.LuceneHSQuery.getQueryHits(LuceneHSQuery.java:360)
 at org.hibernate.search.query.engine.impl.LuceneHSQuery.queryEntityInfos(LuceneHSQuery.java:145)
 at org.hibernate.search.query.hibernate.impl.FullTextQueryImpl.list(FullTextQueryImpl.java:197)
 at org.hibernate.search.jpa.impl.FullTextQueryImpl.getResultList(FullTextQueryImpl.java:157)

堆栈跟踪并不总是相同的(而不是来自readVInt的堆栈跟踪可能位于readVLong中),但是异常始终是ArrayIndexOutOfBoundsException。

当我运行检查索引工具(org.apache.lucene.index.CheckIndex)并将索引从服务器中拉出时,它没有报告任何错误:

Segments file=segments_2 numSegments=1 version=5.5.5 id=e963cq35m2ov726h5wwj8aw39 format=
  1 of 1: name=_1t maxDoc=300616
    version=5.5.5
    id=e963cq35m2ov726h5wwj8aw38
    codec=Lucene54
    compound=false
    numFiles=10
    size (MB)=45.838
    diagnostics = {os=Linux, java.vendor=Oracle Corporation, java.version=1.8.0_141, java.vm.version=25.141-b16, lucene.version=5.5.5, mergeMaxNumSegments=1, os.arch=amd64, java.runtime.version=1.8.0_141-b16, source=merge, mergeFactor=5, os.version=2.6.32-696.3.2.el6.x86_64, timestamp=1533021441077}
    no deletions
    test: open reader.........OK [took 1.027 sec]
    test: check integrity.....OK [took 0.040 sec]
    test: check live docs.....OK [took 0.000 sec]
    test: field infos.........OK [9 fields] [took 0.000 sec]
    test: field norms.........OK [7 fields] [took 0.041 sec]
    test: terms, freq, prox...OK [1231930 terms; 16245644 terms/docs pairs; 16300635 tokens] [took 1.719 sec]
    test: stored fields.......OK [601232 total field count; avg 2.0 fields per doc] [took 0.215 sec]
    test: term vectors........OK [0 total term vector count; avg 0.0 term/freq vector fields per doc] [took 0.000 sec]
    test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET] [took 0.000 sec]

No problems were detected with this index.

我们在主节点上重建索引,然后将其复制到从节点上。我认为这可能是复制方面的损坏问题,但我认为检查程序会报告类似的问题。有什么想法可以解决这个问题?

0 个答案:

没有答案