Question

我有一个使用SOLR数据库的PHP应用程序。当我执行/条款请求（terms doc）

时出现问题

因此，我们感兴趣的文档部分是

poi: "BistriÅ£a",
...
text: [
"ddt",
"Numeric",
"/14/Gagaga 2/11/Economics/17/datenow",
"/20/Daniel_same/11/Economics/17/datenow",
"0/Gagaga 2",
"1/Gagaga 2/Economics",
"2/Gagaga 2/Economics/datenow",
"0/Daniel_same",
"1/Daniel_same/Economics",
"2/Daniel_same/Economics/datenow",
"ppla",
"seat of a first-order administrative division",
"/19/Daniel_same/1071/Plurinational State of Bolivia/2269/Cuba/2272/BistriÅ£a",
"0/Daniel_same",
"1/Daniel_same/Plurinational State of Bolivia",
"2/Daniel_same/Plurinational State of Bolivia/Cuba",
"3/Daniel_same/Plurinational State of Bolivia/Cuba/BistriÅ£a",
"0/Undefined_activity",
"Year",
"0/1999",
"0/1999",
"Measured",
"",
"utf8"
],

请求是

http://localhost:8080/solr/terms
?wt=json
&indent=true
&terms.sort=count
&terms.mincount=1
&terms.limit=10
&terms.regex.flag=case_insensitive
&terms.regex=.*bi.*
&terms.fl=text

回复是

{
    responseHeader: {
        status: 0,
        QTime: 4
    },
    terms: {
        text: [
            "bistriå",
            16
        ]
    }
}

结果的问题是结果文本被截断。我期待着“BistriÅ£a”，这是一个城市Bistrița的UTF-8编码。但结果似乎在特殊字符处被截断。

奇怪的是，如果我使用字段名称“poi”而不是“text”执行请求，我会得到正确的答案

http://localhost:8080/solr/terms
?wt=json
&indent=true
&terms.sort=count
&terms.mincount=1
&terms.limit=10
&terms.regex.flag=case_insensitive
&terms.regex=.*bi.*
&terms.fl=poi

{
    responseHeader: {
        status: 0,
        QTime: 4
    },
    terms: {
        text: [
            "BistriÅ£a",
            16
        ]
    }
}

所以这个词没有被截断。

两个字段之间的最大区别是类型。 Poi 具有字符串类型，而文字具有 text_general 类型。 text_general类型在架构中定义，如此

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <!-- in this example, we will only use synonyms at query time
    <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
    -->
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

如果被问到，我可以提供更多细节。不知道我现在可以添加什么，而不是过多地夸大这个问题。

Answer 1

您可能需要考虑使用text_general字段中的AsciiFoldingFilterFactory来正确处理特殊字符。另外，请参阅Solr提供的可能对您有用的Language Analysis支持。

Solr用/ terms搜索

1 个答案: