在句子标记之前删除python中的多个\ n

时间:2014-10-30 19:00:58

标签: python web-scraping nlp nltk data-cleaning

我是编程的新手,我正在教书和Stack Overflow。我试图在聊天语料库中删除\ n的多个实例,然后对句子进行标记。如果我不删除\ n,字符串将如下所示:

['answers for 10-19-20sUser139 ... hi 10-19-20sUser101 ;)\n\n\n\n\n\n\n\n\n\nI like it when you do it, 10-19-20sUser83\n\n\n\n\n\n\n\n\n\n\n\niamahotnipwithpics\n\n\n\n10-19-20sUser20 go plan the wedding!']

我尝试了几种不同的方法,比如chomps,line,rstrip等,但似乎没有一种方法可行。可能是我错了。整个代码如下所示:

import nltk, re, pprint
from nltk.corpus import nps_chat
chat= nltk.Text(nps_chat.words())
from nltk.corpus import NPSChatCorpusReader
from bs4 import BeautifulSoup
chat=nltk.corpus.nps_chat.raw()
soup= BeautifulSoup(chat)
soup.get_text()
text =soup.get_text()
print(text[:40])
print(len(text))
from nltk.tokenize import sent_tokenize
sent_chat = sent_tokenize(text)
len(sent_chat)
text[:] = [line.rstrip('\n') for line in text]
print(len(sent_chat))
print(sent_chat[:40])

当我使用line方法时,我收到此错误:

Traceback (most recent call last):
File "C:\Python34\Lib\idlelib\testsubjects\sentencelen.py", line 57, in <module>
text[:] = [line.rstrip('\n') for line in text]
TypeError: 'str' object does not support item assignment

帮助?

2 个答案:

答案 0 :(得分:2)

>>> x = 'answers for 10-19-20sUser139 ... hi 10-19-20sUser101 ;)\n\n\n\n\n\n\n\n\n\nI like it when you do it, 10-19-20sUser83\n\n\n\n\n\n\n\n\n\n\n\niamahotnipwithpics\n\n\n\n10-19-20sUser20 go plan the wedding!'
>>> y = "".join([i if i !="\n" else "\t" for i in x])
>>> z = [i for i in y.split('\t') if i]
>>> z
['answers for 10-19-20sUser139 ... hi 10-19-20sUser101 ;)', 'I like it when you do it, 10-19-20sUser83', 'iamahotnipwithpics', '10-19-20sUser20 go plan the wedding!']

答案 1 :(得分:0)

实际上,我偶然发现,如果你先将文字标记为单词,然后将句子标记为\ n就会消失!谢谢你的帮助!