如何在nltk中删除斜杠之前删除POS标签?

时间:2015-11-14 05:49:44

标签: python nlp nltk pos-tagger

这是我的项目的一部分,我需要在这样的短语检测之后表示输出 - (a,x,b)其中a,x,b是短语。我构造了代码并得到了这样的输出:

(CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP))
(CLAUSE (NP Jack/NNP) (VP stayed/VBD) (NP in/IN London/NNP))
(CLAUSE (NP Tom/NNP) (VP is/VBZ) (NP in/IN Kolkata/NNP))

我想让它像前面的表示一样,这意味着我必须删除' CLAUSE',' NP',' VP',' VBD',' NNP'等标签。

怎么做?

我尝试了什么

首先在文本文件中写下这个,标记化并使用list.remove('word')。但这根本没有用。 我正在澄清一点。

我的输入

(CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP)) (CLAUSE (NP Jack/NNP) (VP stayed/VBD) (NP in/IN London/NNP))

输出

<杰克,爱,彼得],[杰克,留在伦敦] 输出只是根据大括号而没有标签。

4 个答案:

答案 0 :(得分:6)

由于您标记了此nltk,因此请使用NLTK的树分析程序处理您的树。我们将在每棵树上阅读,然后简单地打印出树叶。完成。

>>> text ="(CLAUSE (NP Jack/NNP) (VP stayed/VBD) (NP in/IN London/NNP))"
>>> tree = nltk.Tree.fromstring(text, read_leaf=lambda x: x.split("/")[0])
>>> print(tree.leaves())

['Jack', 'stayed', 'in', 'London']

lambda表单拆分每个word/tag对并丢弃标记,只保留单词。

多棵树

我知道,你会问我如何处理整个文件中的这些树,其中一些文件需要不止一行。这是NLTK BracketParseCorpusReader的工作,但它希望终端采用(POS word)而不是word/POS的形式。我不会那么麻烦这样做,因为它更容易欺骗Tree.fromstring()读取你所有的树木,好像它们是一棵树的树枝一样:

allmytext = """
(CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP))
(CLAUSE (NP Jack/NNP) (VP stayed/VBD) (NP in/IN London/NNP))
(CLAUSE (NP Tom/NNP) (VP is/VBZ) (NP in/IN Kolkata/NNP))
"""
wrapped = "(ROOT "+ allmytext + " )"  # Add a "root" node at the top
trees = nltk.Tree.fromstring(wrapped, read_leaf=lambda x: x.split("/")[0])
for tree in trees:
    print(tree.leaves())

如您所见,唯一的区别是我们在文件内容周围添加了"(ROOT "" )",并使用for循环生成输出。循环为我们提供了顶级节点的子节点,即实际的树。

答案 1 :(得分:3)

>>> import re
>>> clause = "(CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP))"
>>> pattern = r'\w+:?(?=\/)'
>>> re.findall(pattern, clause)
['Jack', 'loved', 'Peter']

EDITED

对于多个子句:

>>> import re
>>> pattern = r'\w+:?(?=\/)'
>>> clauses = """(CLAUSE (NP school/NN) (VP is/VBZ situated/VBN) (NP in/IN London/NNP)) (CLAUSE (NP The/DT color/NN of/IN the/DT sky/NN) (VP is/VBZ) (NP pink/NN))"""
>>> [re.findall(pattern, clause) for clause in clauses.split(' (CLAUSE ')]
[['school', 'is', 'situated', 'in', 'London'], ['The', 'color', 'of', 'the', 'sky', 'is', 'pink']]

如果子句用换行符分隔:

>>> import re
>>> pattern = r'\w+:?(?=\/)'
>>> clauses = """(CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP))
... (CLAUSE (NP Jack/NNP) (VP stayed/VBD) (NP in/IN London/NNP))
... (CLAUSE (NP Tom/NNP) (VP is/VBZ) (NP in/IN Kolkata/NNP))"""
>>> [re.findall(pattern, clause) for clause in clauses.split('\n')]
[['Jack', 'loved', 'Peter'], ['Jack', 'stayed', 'in', 'London'], ['Tom', 'is', 'in', 'Kolkata']]

将输出连接到字符串:

>>> " ".join(['Jack', 'loved', 'Peter'])
'Jack loved Peter'

>>> clauses = [['Jack', 'loved', 'Peter'], ['Jack', 'stayed', 'in', 'London'], ['Tom', 'is', 'in', 'Kolkata']]
>>> [" ".join(cl) for cl in clauses]
['Jack loved Peter', 'Jack stayed in London', 'Tom is in Kolkata']

答案 2 :(得分:1)

我正在尝试这样的事情:

import re
tmp = '(CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP))'

tmp = re.split(r'[()/ ]', tmp)
#Use 're.split()' to split by character that was not a letter.
>>> ['', 'CLAUSE', '', 'NP', 'Jack', 'NNP', '', '', 'VP', 'loved', 'VBD', '', '', 'NP', 'Peter', 'NNP', '', '']

result = (tmp[4], tmp[9], tmp[14])
>>> ('Jack', 'loved', 'Peter')

这是你想要的吗?

修改

我应该考虑一下:(。

import re
tmp = '(CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP))'

tmp = re.sub(r'[()]', '', tmp)
>>> 'CLAUSE NP Jack/NNP VP loved/VBD NP Peter/NNP'
result = re.findall(r'[a-zA-Z]*/', tmp)
>>> ['Jack/', 'loved/', 'Peter/']
#Now create a generator.
gen = (i[:-1] for i in result)
tuple(gen)
>>> ('Jack', 'loved', 'Peter')

答案 3 :(得分:0)

当输出为: (CLAUSE (NP Jack/NNP) (VP loved/VBD) (NP Peter/NNP)) (CLAUSE (NP Jack/NNP) (VP stayed/VBD) (NP in/IN London/NNP)) (CLAUSE (NP Tom/NNP) (VP is/VBZ) (NP in/IN Kolkata/NNP)) 很明显它们是树。所以只需使用Tree.leaves()。 这是完整的代码:

def leaves(self):
    """
    Return the leaves of the tree.

        >>> t = Tree.fromstring("(S (NP (D the) (N dog)) (VP (V chased) (NP (D the) (N cat))))")
        >>> t.leaves()
        ['the', 'dog', 'chased', 'the', 'cat']

    :return: a list containing this tree's leaves.
        The order reflects the order of the
        leaves in the tree's hierarchical structure.
    :rtype: list
    """
    leaves = []
    for child in self:
        if isinstance(child, Tree):
            leaves.extend(child.leaves())
        else:
            leaves.append(child)
    return leaves

您可以在此处找到它:http://www.nltk.org/_modules/nltk/tree.html