Question

我有以下代码：

public static void main(String[] args) throws Throwable {
    String[] texts = new String[]{
            "starts_with k mer",
            "starts_with mer",
            "starts_with bleue est mer",
            "starts_with mer est bleue",
            "starts_with mer bla1 bla2 bla3 bla4 bla5",
            "starts_with bleue est la mer",
            "starts_with la mer est bleue",
            "starts_with la mer"
    };


    //write:
    Set<String> stopWords = new HashSet<String>();
    StandardAnalyzer stdAn = new StandardAnalyzer(Version.LUCENE_36, stopWords);
    Directory fsDir = FSDirectory.open(INDEX_DIR);
    IndexWriterConfig iwConf  = new IndexWriterConfig(Version.LUCENE_36,stdAn);
    iwConf.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
    IndexWriter indexWriter = new IndexWriter(fsDir,iwConf);
    for(String text:texts) {
         Document document = new Document();
         document.add(new Field("title",text,Store.YES,Index.ANALYZED));
         indexWriter.addDocument(document);
    }
    indexWriter.commit();

    //read
    IndexReader indexReader = IndexReader.open(fsDir);
    IndexSearcher indexSearcher = new IndexSearcher(indexReader);

    //get query:
    //Query query = getQueryFromString("mer");
    Query query = getQueryFromAPI("mer");

    //explain
    System.out.println("======== Query: "+query+"\n");
    TopDocs hits = indexSearcher.search(query, 10);
    for (ScoreDoc scoreDoc : hits.scoreDocs) {
        Document doc = indexSearcher.doc(scoreDoc.doc);
        System.out.println(">>> "+doc.get("title"));
        System.out.println("Explain:");
        System.out.println(indexSearcher.explain(query, scoreDoc.doc));
    }
}

private static Query getQueryFromString(String searchString) throws Throwable {
    Set<String> stopWords = new HashSet<String>();
    Query query = new QueryParser(Version.LUCENE_36, "title",new StandardAnalyzer(Version.LUCENE_36, stopWords)).parse("("+searchString+") \"STARTS_WITH "+searchString+"\"");
    return query;
}

private static Query getQueryFromAPI(String searchString) throws Throwable {
    Set<String> stopWords = new HashSet<String>();
    Query searchStringTermsMatchTitle = new QueryParser(Version.LUCENE_36, "title", new StandardAnalyzer(Version.LUCENE_36, stopWords)).parse(searchString);

    PhraseQuery titleStartsWithSearchString = new PhraseQuery();
    titleStartsWithSearchString.add(new Term("title","STARTS_WITH".toLowerCase()+" "+searchString));
    BooleanQuery query = new BooleanQuery(true);

    BooleanClause matchClause = new BooleanClause(searchStringTermsMatchTitle, Occur.SHOULD);
    query.add(matchClause);     
    BooleanClause startsWithClause = new BooleanClause(titleStartsWithSearchString, Occur.SHOULD);
    query.add(startsWithClause);

    return query;
}

基本上我正在为一些字符串建立索引，然后我有两种方法可以从用户输入创建Lucene查询，一种只是“手动”（通过字符串连接）构建相应的Lucene查询字符串，另一种使用Lucene的API来创建建立查询。他们似乎正在构建相同的查询，因为查询的调试输出显示完全相同的查询字符串，但搜索结果不一样：

运行通过String连接构建的查询（对于参数“mer”）：

title：mer title：“starts_with mer”

并且在这种情况下，当我用它搜索时，我得到的文档首先与title:"starts_with mer"部分匹配。这是第一个结果的explain：

>>> starts_with mer
Explain:
1.2329358 = (MATCH) sum of:
  0.24658716 = (MATCH) weight(title:mer in 1), product of:
    0.4472136 = queryWeight(title:mer), product of:
      0.882217 = idf(docFreq=8, maxDocs=8)
      0.50692016 = queryNorm
    0.55138564 = (MATCH) fieldWeight(title:mer in 1), product of:
      1.0 = tf(termFreq(title:mer)=1)
      0.882217 = idf(docFreq=8, maxDocs=8)
      0.625 = fieldNorm(field=title, doc=1)
  0.9863486 = (MATCH) weight(title:"starts_with mer" in 1), product of:
    0.8944272 = queryWeight(title:"starts_with mer"), product of:
      1.764434 = idf(title: starts_with=8 mer=8)
      0.50692016 = queryNorm
    1.1027713 = fieldWeight(title:"starts_with mer" in 1), product of:
      1.0 = tf(phraseFreq=1.0)
      1.764434 = idf(title: starts_with=8 mer=8)
      0.625 = fieldNorm(field=title, doc=1)

运行通过Lucene查询助手工具构建的查询会产生一个明显相同的查询：

title：mer title：“starts_with mer”

但这次结果不一样，因为实际上title:"starts_with mer"部分不匹配。这是第一个结果的explain：

>>> starts_with mer
Explain:
0.15185544 = (MATCH) sum of:
  0.15185544 = (MATCH) weight(title:mer in 1), product of:
    0.27540696 = queryWeight(title:mer), product of:
      0.882217 = idf(docFreq=8, maxDocs=8)
      0.312176 = queryNorm
    0.55138564 = (MATCH) fieldWeight(title:mer in 1), product of:
      1.0 = tf(termFreq(title:mer)=1)
      0.882217 = idf(docFreq=8, maxDocs=8)
      0.625 = fieldNorm(field=title, doc=1)

我的问题是：我不能得到同样的结果吗？我真的希望能够在这里使用Query帮助工具，特别是因为我想使用BooleanQuery(disableCoord)选项而且我真的不知道如何直接表达Lucene查询字符串。（是的，我的例子在那里传递“真实”，我也试过“假”，同样的结果）。

=== UPDATE

femtoRgon的答案很棒：问题在于我将整个搜索字符串添加为术语，而不是首先将其拆分为术语，然后将每个搜索字符串添加到查询中。

如果输入字符串由一个术语组成，则femtoRgon的答案可以正常工作：在这种情况下，将“STARTS_WITH”文本分开添加为一个术语，然后将搜索字符串添加为第二个术语。

但是，如果用户输入的内容将被多个术语标记，则必须先将其拆分为术语（最好使用索引时使用的相同分析器和/或标记器 - 以获得一致的结果）然后将每个术语添加到查询中。

我最终做的是创建一个函数，使用我用于编制索引的相同分析器将查询字符串拆分为术语：

private static List<String> getTerms(String text) throws Throwable {
    Analyzer analyzer = getAnalyzer();      
    StringReader textReader = new StringReader(text);
    TokenStream tokenStream = analyzer.tokenStream(FIELD_NAME_TITLE, textReader);
    tokenStream.reset();        
    List<String> terms = new ArrayList<String>();
    CharTermAttribute charTermAttribute = tokenStream.addAttribute(CharTermAttribute.class);
    while (tokenStream.incrementToken()) {
        String term = charTermAttribute.toString();
        terms.add(term);
    }
    textReader.close();
    tokenStream.close();
    analyzer.close();       
    return terms;
}

然后我首先将“STARTS_WITH”添加为一个术语，然后将列表中的每个元素作为单独的术语添加：

PhraseQuery titleStartsWithSearchString = new PhraseQuery();
titleStartsWithSearchString.add(new Term("title","STARTS_WITH".toLowerCase()));
for(String term:getTerms(searchString)) {
    titleStartsWithSearchString.add(new Term("title",term));
}

Answer 1

我相信您遇到的问题是您将整个短语作为单个术语添加到PhraseQuery中。在索引和QueryParser解析的查询中，这将被分为"starts_with"和"mer"，必须连续找到。但是，在您构建的查询中，您在PhraseQuery中只有一个术语，即术语"starts_with mer"，它在索引中不作为单个术语存在。

您应该能够将构建PhraseQuery的位更改为：

PhraseQuery titleStartsWithSearchString = new PhraseQuery();
titleStartsWithSearchString.add(new Term("title","STARTS_WITH".toLowerCase())
titleStartsWithSearchString.add(new Term("title",searchString));

Lucene：从String解析并通过Query API构建的相同查询不会产生相同的结果

1 个答案: