Question

NLTK附带一些语料库样本： http://nltk.googlecode.com/svn/trunk/nltk_data/index.xml

我想只有没有编码的文字。我不知道如何提取这样的内容。我要提取的是

1）nps_chat：解压缩后，filename就像10-19-20s_706posts.xml。这样的文件是XML格式，如：

<Posts>
<Post class="Statement" user="10-19-20sUser7">now im left with this gay name<terminals>

                <t pos="RB" word="now"/>
                <t pos="PRP" word="im"/>
                <t pos="VBD" word="left"/>
                <t pos="IN" word="with"/>
                <t pos="DT" word="this"/>
                <t pos="JJ" word="gay"/>
                <t pos="NN" word="name"/>
            </terminals>

        </Post>
            ...
            ...

我只想要那个实际的帖子：

now im left with this gay name

如何在NLTK或（无论如何）在本地磁盘中删除编码后保存裸帖？

2）交换机成绩单。这种类型的文件（文件名是解压缩后的话语）包含以下格式。我想要的是剥离前面的标记：

o A.1 utt1: Okay, /
qy A.1 utt2: have you ever served as a juror? /
ng B.2 utt1: Never. /
sd^e B.2 utt2: I've never been served on the jury, never been called up in a jury, although some of my friends have been jurors. /
b A.3 utt1: Uh-huh. /
sd A.3 utt2: I never have either. /
% B.4 utt1: You haven't, {F huh. } /
...
...

我想只有：

Okay, /
have you ever served as a juror? /
Never. /
I've never been served on the jury, never been called up in a jury, although some of my friends have been jurors. /
Uh-huh. /
I never have either. /
You haven't, {F huh. } /
...
...

非常感谢你。

Answer 1

首先，您需要为语料库创建corpus reader。您可以在nltk.corpus中使用一些语料库阅读器，例如：

AlpinoCorpusReader
BNCCorpusReader
BracketParseCorpusReader
CMUDictCorpusReader
CategorizedCorpusReader
CategorizedPlaintextCorpusReader
CategorizedTaggedCorpusReader
ChunkedCorpusReader
ConllChunkCorpusReader
ConllCorpusReader
CorpusReader
DependencyCorpusReader
EuroparlCorpusReader
IEERCorpusReader
IPIPANCorpusReader
IndianCorpusReader
MacMorphoCorpusReader
NPSChatCorpusReader
NombankCorpusReader
PPAttachmentCorpusReader
Pl196xCorpusReader
PlaintextCorpusReader
PortugueseCategorizedPlaintextCorpusReader
PropbankCorpusReader
RTECorpusReader
SensevalCorpusReader
SinicaTreebankCorpusReader
StringCategoryCorpusReader
SwadeshCorpusReader
SwitchboardCorpusReader
SyntaxCorpusReader
TaggedCorpusReader
TimitCorpusReader
ToolboxCorpusReader
VerbnetCorpusReader
WordListCorpusReader
WordNetCorpusReader
WordNetICCorpusReader
XMLCorpusReader
YCOECorpusReader

一旦你从语料库中创建了一个语料库阅读器，就像这样：

c = nltk.corpus.whateverCorpusReaderYouChoose(directoryWithCorpus, regexForFileTypes)

您可以使用以下代码从语料库中获取单词：

paragraphs = [para for para in c.paras()]
for para in paragraphs:
    words = [word for sentence in para for word in sentence]

这可以为您提供语料库所有段落中所有单词的列表。

希望这有帮助

Answer 2

您可以使用nltk语料库中的Function NumOut(strIn As String) As String Dim rng1 As Range Set rng1 = Range("A" & Join(Split(Application.Trim([a1]), ", "), ",A")) 'force the range into areas rather than cells Set rng1 = Union(rng1, rng1) NumOut = Replace(Replace(Replace(rng1.Address, "$A$", vbNullstring), ": ", "-"), ",", ", ") End Function属性

.words()

这将为您提供列表中的所有单词

content = nps_chat.words()

如何从NLTK附带的样本语料库中提取单词？

2 个答案: