通过比较python中的多个列表来删除/列出重复项

时间:2014-05-14 02:29:20

标签: python list duplicates

我知道已经询问了列表中的删除/列表重复项。我在采用同时比较多个列表时遇到问题。

lst = [item1, item2, item3, item4, item5]
a = [1,2,1,5,1]
b = [2,0,2,5,2]
c = [0,1,0,1,5]

如果这些是我的列表,我想比较它们就好像我使用的是zip功能。我想检查是否列表中的索引0,2和4是重复的,如果那些相同的索引是其他列表的重复,例如在列表b 0,2和4也是重复,但在列表中c 0和2是唯一的重复因此,我只想从lst列出索引0和2获得结果列表[item1,item3]

我如何采用这种def来做到这一点?

def list_duplicates(seq):
  seen = set()
  seen_add = seen.add
  # adds all elements it doesn't know yet to seen and all other to seen_twice
  seen_twice = set( x for x in seq if x in seen or seen_add(x) )
  # turn the set into a list (as requested)
  return list( seen_twice )

a = [1,2,3,2,1,5,6,5,5,5]
list_duplicates(a) # yields [1, 2, 5]

2 个答案:

答案 0 :(得分:0)

您正在尝试确定哪些公共索引包含多个列表中的重复值,而不是跟踪重复值本身。这意味着除了跟踪给定seq中重复的项目之外,我们还需要跟踪找到重复项目的索引。这是添加到现有方法的非常直接的方法:

from collections import defaultdict

def list_duplicates(seq):
    seen = set()
    seen_twice = set()
    seen_indices = defaultdict(list)  # To keep track of seen indices
    for index, x in enumerate(seq):  # Can't use a comprehension now, too much logic in there.
        seen_indices[x].append(index)
        if x in seen:
            seen_twice.add(val)
        else:
            seen.add(val)
    print seen_indices
    return list( seen_twice )

if __name__ == "__main__":
    a = [1,2,3,2,1,5,6,5,5,5]   
    duped_items = list_duplicates(a)
    print duped_items

输出:

defaultdict(<type 'list'>, {1: [0, 4], 2: [1, 3], 3: [2], 5: [5, 7, 8, 9], 6: [6]})
[1, 2, 5]

所以现在除了欺骗值本身之外,我们还会跟踪所有欺骗值的指数。

下一步是以某种方式在多个列表中应用它。我们可以利用这样一个事实:一旦我们遍历一个列表,我们就会消除一堆我们知道并不指向重复值的索引,并且只会迭代后续列表中的已知重复索引。这需要重新修改逻辑,迭代“可能重复的索引”而不是整个列表:

def list_duplicates2(*seqs):
    val_range = range(0, len(seqs[0]))  # At first, all indices could be duplicates.
    for seq in seqs:
        # Set up is the same as before.
        seen_items = set()
        seen_twice = set()
        seen_indices = defaultdict(list)
        for index in val_range:  # Iterate over the possibly duplicated indices, not the  whole sequence
            val = seq[index]
            seen_indices[val].append(index)
            if val in seen_items:
                seen_twice.add(val)
            else:
                seen_items.add(val)
        # Now that we've gone over the current valid_range, we can create a
        # new valid_range for the next iteration by only including the indices
        # in seq which contained values that we found at least twice in the 
        # current valid_range.
        val_range = [duped_index for seen_val in seen_twice for duped_index in seen_indices[seen_val]]
        print "new val_range is %s" % val_range
    return val_range

if __name__ == "__main__":
    a = [1,2,1,5,1]
    b = [2,0,2,5,2]
    c = [0,1,0,1,5]
    duped_indices = list_duplicates2(a, b, c)
    print "duped_indices is %s" % duped_indices

输出:

new val_range is [0, 2, 4]
new val_range is [0, 2, 4]
new val_range is [0, 2]
duped_indices is [0, 2]

这正是你想要的。

答案 1 :(得分:0)

在此列表中搜索重复项

l = [[a[i],b[i],c[i]] for i in range(len(a))]

对于您的示例,它将生成此列表:

[[1, 2, 0], [2, 0, 1], [1, 2, 0], [5, 5, 1], [1, 2, 5]]

然后:

result = [lst[i] for (i,x) in enumerate(l) if x in list_duplicates(l)]