如果列表中包含来自“黑名单”的子字符串,则从列表中删除该项

时间:2014-11-14 14:51:18

标签: python regex string list-comprehension

在python中,我想从列表中删除任何包含在所谓的"黑名单"中找到的子字符串的字符串。

例如,假设列表A如下:

A = [ 'cat', 'doXXXg', 'monkey', 'hoBBBrse', 'fish', 'snake']

和列表B是:

B = ['XXX', 'BBB']

我怎么能得到列表C:

C = [ 'cat', 'monkey', 'fish', 'snake']

我已经使用了正则表达式和列表推导的各种组合,但我似乎无法让它工作。

2 个答案:

答案 0 :(得分:8)

>>> A = [ 'cat', 'doXXXg', 'monkey', 'hoBBBrse', 'fish', 'snake']
>>> B = ['XXX', 'BBB']

以下列表理解将起作用

>>> [word for word in A if not any(bad in word for bad in B)]
['cat', 'monkey', 'fish', 'snake']

答案 1 :(得分:6)

您可以将黑名单加入一个表达式:

import re

blacklist = re.compile('|'.join([re.escape(word) for word in B]))

然后如果匹配则过滤掉单词:

C = [word for word in A if not blacklist.search(word)]

模式中的单词被转义(因此.和其他元字符不被视为这样,而是作为文字字符),并加入一系列|替代品:

>>> '|'.join([re.escape(word) for word in B])
'XXX|BBB'

演示:

>>> import re
>>> A = [ 'cat', 'doXXXg', 'monkey', 'hoBBBrse', 'fish', 'snake']
>>> B = ['XXX', 'BBB']
>>> blacklist = re.compile('|'.join([re.escape(word) for word in B]))
>>> [word for word in A if not blacklist.search(word)]
['cat', 'monkey', 'fish', 'snake']

这应该胜过任何明确的成员资格测试,特别是当黑名单中的单词数量增加时:

>>> import string, random, timeit
>>> def regex_filter(words, blacklist):
...     [word for word in A if not blacklist.search(word)]
... 
>>> def any_filter(words, blacklist):
...     [word for word in A if not any(bad in word for bad in B)]
... 
>>> words = [''.join([random.choice(string.letters) for _ in range(random.randint(3, 20))])
...          for _ in range(1000)]
>>> blacklist = [''.join([random.choice(string.letters) for _ in range(random.randint(2, 5))])
...              for _ in range(10)]
>>> timeit.timeit('any_filter(words, blacklist)', 'from __main__ import any_filter, words, blacklist', number=100000)
0.36232495307922363
>>> timeit.timeit('regex_filter(words, blacklist)', "from __main__ import re, regex_filter, words, blacklist; blacklist = re.compile('|'.join([re.escape(word) for word in blacklist]))", number=100000)
0.2499098777770996

以上测试10个随机黑名单短字(2 - 5个字符)对1000个随机单词列表(3 - 20个字符长),正则表达式快50%。