R-NLP-文字清理

时间:2018-07-08 14:52:32

标签: r nlp text-mining

我是文本挖掘的新手,目前,我坚持使用这种模式

pattern = c(
    "<f0><U+009F><U+0098><U+00AD>", 
    "<f0><U+009F><U+0099><U+008F>",
    "<f0><U+009F><U+008F><U+00BF> ",
    "<f0><U+009F><U+0098><U+0082>", 
    " <f0><U+009F><U+00A4><U+00B7>",
    "  <f0><U+009F><U+008F><U+00BD><U+200D><U+2640><U+FE0F>\r\nBody",
    " <f0><U+009F><U+00A4><U+00A3>", 
    " <f0><U+009F><U+0099><U+0084> ", 
    "  <f0><U+009F><U+0099><U+0084>",
    "  <f0><U+009F><U+0099><U+0083>",
      "<f0><U+009F><U+0098><U+00B4>",
     "Hello")

我只想接收pattern =“ Hello”,并排除所有其他文本。

我尝试了以下操作,但立即失败:

gsub(c, "<f0><U+00F><U+[0-9]><U+[a-zA-Z0-9]>*, replacement = "")

所以,我试图将其分解:

a = gsub(c, pattern = "<f0>", replacement = "")

->结果<fo>下降,所以这是一个好兆头,但是当我下一步时

gsub(a, pattern = "<U+009F>", replacement = "")

->结果:保留<U+009F>。 你有什么主意吗? 我感谢任何建议! 预先感谢!

1 个答案:

答案 0 :(得分:1)

两种清除文本的方法。没有给出允许删除“身体”的标准。

x <- pattern # to avoid ambiguity in function parameters

# by finding words longer than two letters (so not 'a' or 'I' either)
words <- unlist(regmatches(x, gregexpr("\\b[[:alpha:]]{2,}\\b", x, perl=TRUE)))
words

#[1] "Body"  "Hello"

# by removing unwanted characters and character sequences
cleaned <- gsub("(<[^>]{0,}>|\\r|\\n)", "", x, perl=TRUE)
# and removing leading and trailing spaces
cleaned <- gsub("^ {1,}| {1,}$", "", cleaned, perl=TRUE)
cleaned[cleaned != ""]

#[1] "Body"  "Hello"