ElasticSearch找不到UTF8字符

时间:2015-01-24 12:43:06

标签: symfony elasticsearch foselasticabundle

我正在使用ElasticSearch以及Symfony2和FOSElasticaBundle。然后我尝试找到只有ASCII字符的东西,它就像一个魅力。此外,我正在使用JSON数组进行数据处理,并将unicode字符转换为类似\ u00f8C的字符。这会影响我的问题吗?我的搜索代码:

function __construct(Request $request, TransformedFinder $artist_finder, TransformedFinder $picture_finder, Router $router)
{
    $this->request = $request;
    $this->a_finder = $artist_finder;
    $this->p_finder = $picture_finder;
    $this->router = $router;
}

/**
 * @param $query
 * @return array
 */
public function search($query)
{
    $locale = $this->request->getLocale();

    $artist_results = $this->a_finder->find($query.'*', self::SEARCH_LIMIT);
    $gallery_results = $this->p_finder->find($query.'*', self::SEARCH_LIMIT);

    $results = array(
        'artist' => array(),
        'gallery' => array()
    );

    foreach($artist_results as $a)
    {
        $results['artist'][] = array(
            'url' => $this->router->generate('artist_page', ['slug' => $a->getSlug()]),
            'name' => $a->getInfo()[$locale]['name']. ' ' .$a->getInfo()[$locale]['surname']
        );
    }

    foreach($gallery_results as $a)
    {
        $results['gallery'][] = array(
            'url' => $this->router->generate('picture_page', ['slug' => $a->getSlug()]),
            'name' => $a->getInfo()[$locale]['title']
        );
    }

    return $results;
}

此外,ES是否有能力看到立陶宛字符之间没有区别,例如“Š”和“S”,“Ė”和“E”等?我的意思是,我希望得到与“žąsis”和“zasis”等查询相同的结果。

3 个答案:

答案 0 :(得分:2)

我设法让它发挥作用。我必须使用第二个json_encode参数JSON_UNESCAPED_UNICODE作为索引值,我认为这是有道理的。

答案 1 :(得分:1)

您必须配置analyser section of your fos_elastica bundle configuration

根据您的需要,可以应用不同的分析仪。并且有特定的语言,请查看相关的elasticsearch documentation

答案 2 :(得分:1)

你需要https://github.com/elasticsearch/elasticsearch-analysis-icu插件和一些过滤器配置(在我的情况下是捷克语)...例如。

settings:
    analysis:
      analyzer:
        cs_collation:
          type: custom
          tokenizer: keyword
          filter: [trim,cs_icu_collation]
        cs_facets:
          type: custom
          tokenizer: keyword
          filter: [trim]
        custom_analyzer_cs:
          type     :    custom
          tokenizer:    nGram
          filter   :    [stopwords, lowercase, czech_stemmer, asciifolding, snowball, worddelimiter]


filter:
    czech_stemmer:
      type: stemmer
      name: czech
    cs_icu_collation:
      type: icu_collation
      stopwords: [_czech_]
      language: cs