UTF-8规范化Logstash到Elasticsearch

时间:2016-02-26 16:23:32

标签: elasticsearch logstash

数据标准规定进入ES索引的所有数据必须为UTF-8。我正在寻找一个logstash组件,它允许我应用某种UTF-8过滤器来规范化数据流。

1 个答案:

答案 0 :(得分:0)

好的,所以你通过UDP获得ISO数据(你无法改变),你想以某种方式将其转换为UTF8。

嗯,只要知道Logstash已经将您的输入转换为UTF-8,您只是告诉它您通过$str = 'type == \'prova\' && padposition == "stefano" or 10>var_name'; $pattern = <<<'EOD' ~ # you define first the basic elements (as for a lexer) with named groups (?(DEFINE) (?<var> [a-z_]\w* ) # variable name (?<dqstr> (?<=") [^\\"]*+ (?s:\\.[^\\"]*)*+ (?=") ) # double quoted string (?<sqstr> (?<=') [^\\']*+ (?s:\\.[^\\']*)*+ (?=') ) # single quoted string (?<string> \g<dqstr> | \g<sqstr> ) # any string (?<num> [0-9]+(?:\.[0-9]+)? ) # a number (?<value> \g<string> | \g<num> ) # any value (?<comp> [!><=]= | =?[><] ) # comparison operator ) # Then you write the pattern using these named groups (?J) # allow duplicate named groups # variable op value (?<key> \g<var> ) \h* \g<comp> \h* ["']? (?<val> \g<value> ) ['"]? | # OR # value op variable ["']? (?<val> \g<value> ) ['"]? \h* \g<comp> \h* (?<key> \g<var> ) ~xi EOD; if (preg_match_all($pattern, $str, $matches, PREG_SET_ORDER)) { $arr_name = []; foreach($matches as $m) { $arr_name[$m['key']] = $m['val']; } print_r($arr_name); } 获得了一些ISO编码的内容,但Logstash会处理UTF-8下游的一切。

您可以查看此主题以获取更多信息:Building Math.NET Numerics