如何从列表列表中过滤特定的POS标签到单独的列表？

时间：2017-05-16 14:20:38

标签： python python-3.x nlp pos-tagger

我有大量的产品描述数据，需要将产品名称和意图与描述分开，我发现在使用POS标签标记文本后分离NNP标签对于进一步清理有一定帮助。

我有以下类似的数据，我想只过滤NNP标记，并希望它们在各自的列表中进行过滤，但无法这样做。 

 data = [[('User', 'NNP'),
  ('is', 'VBZ'),
  ('not', 'RB'),
  ('able', 'JJ'),
  ('to', 'TO'),
  ('order', 'NN'),
  ('products', 'NNS'),
  ('from', 'IN'),
  ('iShopCatalog', 'NN'),
  ('Coala', 'NNP'),
  ('excluding', 'VBG'),
  ('articles', 'NNS'),
  ('from', 'IN'),
  ('VWR', 'NNP')],
 [('Arfter', 'NNP'),
  ('transferring', 'VBG'),
  ('the', 'DT'),
  ('articles', 'NNS'),
  ('from', 'IN'),
  ('COALA', 'NNP'),
  ('to', 'TO'),
  ('SRM', 'VB'),
  ('the', 'DT'),
  ('Category', 'NNP'),
  ('S9901', 'NNP'),
  ('Dummy', 'NNP'),
  ('is', 'VBZ'),
  ('maintained', 'VBN')],
 [('Due', 'JJ'),
  ('to', 'TO'),
  ('this', 'DT'),
  ('the', 'DT'),
  ('user', 'NN'),
  ('is', 'VBZ'),
  ('not', 'RB'),
  ('able', 'JJ'),
  ('to', 'TO'),
  ('order', 'NN'),
  ('the', 'DT'),
  ('product', 'NN')],
 [('All', 'DT'),
  ('other', 'JJ'),
  ('users', 'NNS'),
  ('can', 'MD'),
  ('order', 'NN'),
  ('these', 'DT'),
  ('articles', 'NNS')],
 [('She', 'PRP'),
  ('can', 'MD'),
  ('order', 'NN'),
  ('other', 'JJ'),
  ('products', 'NNS'),
  ('from', 'IN'),
  ('a', 'DT'),
  ('POETcatalog', 'NNP'),
  ('without', 'IN'),
  ('any', 'DT'),
  ('problems', 'NNS')],
 [('Furtheremore', 'IN'),
  ('she', 'PRP'),
  ('is', 'VBZ'),
  ('able', 'JJ'),
  ('to', 'TO'),
  ('order', 'NN'),
  ('products', 'NNS'),
  ('from', 'IN'),
  ('the', 'DT'),
  ('Vendor', 'NNP'),
  ('VWR', 'NNP'),
  ('through', 'IN'),
  ('COALA', 'NNP')],
 [('But', 'CC'),
  ('articles', 'NNS'),
  ('from', 'IN'),
  ('all', 'DT'),
  ('other', 'JJ'),
  ('suppliers', 'NNS'),
  ('are', 'VBP'),
  ('not', 'RB'),
  ('orderable', 'JJ')],
 [('I', 'PRP'),
  ('already', 'RB'),
  ('spoke', 'VBD'),
  ('to', 'TO'),
  ('anic', 'VB'),
  ('who', 'WP'),
  ('maintain', 'VBP'),
  ('the', 'DT'),
  ('catalog', 'NN'),
  ('COALA', 'NNP'),
  ('and', 'CC'),
  ('they', 'PRP'),
  ('said', 'VBD'),
  ('that', 'IN'),
  ('the', 'DT'),
  ('reason', 'NN'),
  ('should', 'MD'),
  ('be', 'VB'),
  ('the', 'DT'),
  ('assignment', 'NN'),
  ('of', 'IN'),
  ('the', 'DT'),
  ('plant', 'NN')],
 [('User', 'NNP'),
  ('is', 'VBZ'),
  ('a', 'DT'),
  ('assinged', 'JJ'),
  ('to', 'TO'),
  ('Universitaet', 'NNP'),
  ('Regensburg', 'NNP'),
  ('in', 'IN'),
  ('Scout', 'NNP'),
  ('but', 'CC'),
  ('in', 'IN'),
  ('P17', 'NNP'),
  ('table', 'NN'),
  ('YESRMCDMUSER01', 'NNP'),
  ('she', 'PRP'),
  ('is', 'VBZ'),
  ('assigned', 'VBN'),
  ('to', 'TO'),
  ('company', 'NN'),
  ('001500', 'CD'),
  ('Merck', 'NNP'),
  ('KGaA', 'NNP')],
 [('Please', 'NNP'),
  ('find', 'VB'),
  ('attached', 'JJ'),
  ('some', 'DT'),
  ('screenshots', 'NNS')]]

我写了以下代码：

def prodname(a):
    p = []
    for i in a:
        for j in range(len(i)):
            if i[j][1]=='NNP':
                p.append(i[j][0])
    return p

提供以下输出：

    ['User',
     'Coala',
     'VWR',
     'Arfter',
     'COALA',
     'Category',
     'S9901',
     'Dummy',
     'POETcatalog',
     'Vendor',
     'VWR',
     'COALA',
     'COALA',
     'User',
     'Universitaet',
     'Regensburg',
     'Scout',
     'P17',
     'YESRMCDMUSER01',
     'Merck',
     'KGaA',
     'Please']

我想得到的输出是：

[['User',
  'Coala',
  'VWR']
['Arfter',
 'COALA',
 'Category',
 'S9901',
 'Dummy']
[],
[],
['POETcatalog'],
['Vendor',
 'VWR',
 'COALA'],
[],
['COALA'],
['User',
 'Universitaet',
 'Regensburg',
 'Scout',
 'P17',
 'YESRMCDMUSER01',
 'Merck',
'KGaA'],
['Please']]

还尝试使用[[] for i in range(len(data)]附加到各自的列表，但不能这样做。

2 个答案:

答案 0 :(得分：1)

您可以使用此列表理解：

[[j[0] for j in i if j[-1]=="NNP"] for i in data]

输出：

[['User', 'Coala', 'VWR'], ['Arfter', 'COALA', 'Category', 'S9901', 'Dummy'], [], [], ['POETcatalog'], ['Vendor', 'VWR', 'COALA'], [], ['COALA'], ['User', 'Universitaet', 'Regensburg', 'Scout', 'P17', 'YESRMCDMUSER01', 'Merck', 'KGaA'], ['Please']]

答案 1 :(得分：1)

列表理解是要走的路。但是@McGrady的回答可能有点难以阅读。

这是一个更易于阅读的解决方案：

document = [[('User', 'NNP'), ('is', 'VBZ'), ('not', 'RB'), ('able', 'JJ'), ('to', 'TO'), ('order', 'NN'), ('products', 'NNS'), ('from', 'IN'), ('iShopCatalog', 'NN'), ('Coala', 'NNP'), ('excluding', 'VBG'), ('articles', 'NNS'), ('from', 'IN'), ('VWR', 'NNP')], [('Arfter', 'NNP'), ('transferring', 'VBG'), ('the', 'DT'), ('articles', 'NNS'), ('from', 'IN'), ('COALA', 'NNP'), ('to', 'TO'), ('SRM', 'VB'), ('the', 'DT'), ('Category', 'NNP'), ('S9901', 'NNP'), ('Dummy', 'NNP'), ('is', 'VBZ'), ('maintained', 'VBN')], [('Due', 'JJ'), ('to', 'TO'), ('this', 'DT'), ('the', 'DT'), ('user', 'NN'), ('is', 'VBZ'), ('not', 'RB'), ('able', 'JJ'), ('to', 'TO'), ('order', 'NN'), ('the', 'DT'), ('product', 'NN')], [('All', 'DT'), ('other', 'JJ'), ('users', 'NNS'), ('can', 'MD'), ('order', 'NN'), ('these', 'DT'), ('articles', 'NNS')], [('She', 'PRP'), ('can', 'MD'), ('order', 'NN'), ('other', 'JJ'), ('products', 'NNS'), ('from', 'IN'), ('a', 'DT'), ('POETcatalog', 'NNP'), ('without', 'IN'), ('any', 'DT'), ('problems', 'NNS')], [('Furtheremore', 'IN'), ('she', 'PRP'), ('is', 'VBZ'), ('able', 'JJ'), ('to', 'TO'), ('order', 'NN'), ('products', 'NNS'), ('from', 'IN'), ('the', 'DT'), ('Vendor', 'NNP'), ('VWR', 'NNP'), ('through', 'IN'), ('COALA', 'NNP')], [('But', 'CC'), ('articles', 'NNS'), ('from', 'IN'), ('all', 'DT'), ('other', 'JJ'), ('suppliers', 'NNS'), ('are', 'VBP'), ('not', 'RB'), ('orderable', 'JJ')], [('I', 'PRP'), ('already', 'RB'), ('spoke', 'VBD'), ('to', 'TO'), ('anic', 'VB'), ('who', 'WP'), ('maintain', 'VBP'), ('the', 'DT'), ('catalog', 'NN'), ('COALA', 'NNP'), ('and', 'CC'), ('they', 'PRP'), ('said', 'VBD'), ('that', 'IN'), ('the', 'DT'), ('reason', 'NN'), ('should', 'MD'), ('be', 'VB'), ('the', 'DT'), ('assignment', 'NN'), ('of', 'IN'), ('the', 'DT'), ('plant', 'NN')], [('User', 'NNP'), ('is', 'VBZ'), ('a', 'DT'), ('assinged', 'JJ'), ('to', 'TO'), ('Universitaet', 'NNP'), ('Regensburg', 'NNP'), ('in', 'IN'), ('Scout', 'NNP'), ('but', 'CC'), ('in', 'IN'), ('P17', 'NNP'), ('table', 'NN'), ('YESRMCDMUSER01', 'NNP'), ('she', 'PRP'), ('is', 'VBZ'), ('assigned', 'VBN'), ('to', 'TO'), ('company', 'NN'), ('001500', 'CD'), ('Merck', 'NNP'), ('KGaA', 'NNP')], [('Please', 'NNP'), ('find', 'VB'), ('attached', 'JJ'), ('some', 'DT'), ('screenshots', 'NNS')]]
output = [[word for word, pos in sentence if pos=='NNP'] for sentence in document]

如果你喜欢更清晰的代码，你可以围绕嵌套列表理解，https://stackoverflow.com/a/3633145/610569：

output = [word for sentence in document for word, pos in sentence if pos=='NNP']
相关问题
最新问题