Question

我正在尝试使用RTextTools附带的德语词干分析器，但我得到的结果非常不合适。

说，我有以下向量：

v <- c("groß", "größer", "am", "größten", "ähnlicher")

使用

library(RTextTools)
wordStem(v, "german")

我得到了

[1] "groß"    "größer"  "am"      "größten" "ähnlich"

我错过了什么？

Answer 1

Snowball中的算法

/*
    Extra rule for -nisse ending added 11 Dec 2009
*/

routines (
           prelude postlude
           mark_regions
           R1 R2
           standard_suffix
)

externals ( stem )

integers ( p1 p2 x )

groupings ( v s_ending st_ending )

stringescapes {}

/* special characters (in ISO Latin I) */

stringdef a"   hex 'E4'
stringdef o"   hex 'F6'
stringdef u"   hex 'FC'
stringdef ss   hex 'DF'
......

看起来好像被翻译回'DF'“ß”

通过以下e表示变音符号德语字母ä，ö和ü分别由ae，oe和ue代表。这里的词干分析器是德国主要词干分析器的变体，可以考虑到这一点。

主要的德国词干开始于规则，

First, replace ß by ss, and put u and y between vowels into upper case.

这被替换为规则

Put u and y between vowels into upper case, and then do the following mappings,

    (a) replace ß with ss, **"MAYBE WRONG ORDER"**
    (a) replace ae with ä,
    (a) replace oe with ö,
    (a) replace ue with ü unless preceded by q. 



So in quelle, ue is not mapped to ü because it follows q, and in feuer it is not mapped because the first part of the rule changes it to feUer, so the u is not found.

RTextTools中的德国词干分析器

1 个答案: