Question

在没有IList

的 C＃中删除Linq中重复项的最有效方法是什么？

我有来自另一个SO [1]的以下代码，

IList<IList<int>> output = new List<IList<int>>(); 
var lists = output;
for (int i = 0; i < lists.Count; ++i)
{
  //since we want to compare sequecnes, we shall ensure the same order of the items
   var item = lists[i].OrderBy(x => x).ToArray();
   for (int j = lists.Count - 1; j > i; --j)
        if (item.SequenceEqual(lists[j].OrderBy(x => x)))
          lists.RemoveAt(j);
 }

我在更大的编码挑战中使用它，没有Linq或语法糖，我试图看看是否有任何优雅/快速的解决方案？

我在考虑使用Hash，但我不确定使用什么类型的Hashing函数来确定List已经可用？

更清楚对于像

这样的输入

{{1,2,4, 4}, {3,4,5}, {4,2,1,4} }

中间输出是[排序输入/输出正常]

{{1,2,4,4}, {3,4,5}, {1,2,4,4} }

输出：

{{1,2,4,4}, {3,4,5}}

Answer 1

我使用了Microsoft的CollectionAssert.AreEquivalent内部的修改版本：

using System.Collections.Generic;

public class Program
{
    public static void Main()
    {
        var lists = new List<List<int>>
        {
            new List<int> {1, 4, 2},
            new List<int> {3, 4, 5},
            new List<int> {1, 2, 4}
        };

        var dedupe =
            new List<List<int>>(new HashSet<List<int>>(lists, new MultiSetComparer<int>()));
    }

    // Equal if sequence contains the same number of items, in any order
    public class MultiSetComparer<T> : IEqualityComparer<IEnumerable<T>>
    {
        public bool Equals(IEnumerable<T> first, IEnumerable<T> second)
        {
            if (first == null)
                return second == null;

            if (second == null)
                return false;

            if (ReferenceEquals(first, second))
                return true;

            // Shortcut when we can cheaply look at counts
            var firstCollection = first as ICollection<T>;
            var secondCollection = second as ICollection<T>;
            if (firstCollection != null && secondCollection != null)
            {
                if (firstCollection.Count != secondCollection.Count)
                    return false;

                if (firstCollection.Count == 0)
                    return true;
            }

            // Now compare elements
            return !HaveMismatchedElement(first, second);
        }

        private static bool HaveMismatchedElement(IEnumerable<T> first, IEnumerable<T> second)
        {
            int firstNullCount;
            int secondNullCount;

            // Create dictionary of unique elements with their counts
            var firstElementCounts = GetElementCounts(first, out firstNullCount);
            var secondElementCounts = GetElementCounts(second, out secondNullCount);

            if (firstNullCount != secondNullCount || firstElementCounts.Count != secondElementCounts.Count)
                return true;

            // make sure the counts for each element are equal, exiting early as soon as they differ
            foreach (var kvp in firstElementCounts)
            {
                var firstElementCount = kvp.Value;
                int secondElementCount;
                secondElementCounts.TryGetValue(kvp.Key, out secondElementCount);

                if (firstElementCount != secondElementCount)
                    return true;
            }

            return false;
        }

        private static Dictionary<T, int> GetElementCounts(IEnumerable<T> enumerable, out int nullCount)
        {
            var dictionary = new Dictionary<T, int>();
            nullCount = 0;

            foreach (T element in enumerable)
            {
                if (element == null)
                {
                    nullCount++;
                }
                else
                {
                    int num;
                    dictionary.TryGetValue(element, out num);
                    num++;
                    dictionary[element] = num;
                }
            }

            return dictionary;
        }

        public int GetHashCode(IEnumerable<T> enumerable)
        {
            int hash = 17;
            // Create and sort list in-place, rather than OrderBy(x=>x), because linq is forbidden in this question
            var list = new List<T>(enumerable);
            list.Sort();
            foreach (T val in list)
                hash = hash * 23 + (val == null ? 42 : val.GetHashCode());

            return hash;
        }
    }
}

这使用Hashset<T>，添加到此集合会自动忽略重复项。

最后一行可以是：

var dedupe = new HashSet<List<int>>(lists, new MultiSetComparer<int>()).ToList();

从技术上讲，它使用System.Linq命名空间，但我不认为这是您对Linq的关注。

我将回应Eric Lippert所说的话。您要求我们向您展示Linq和框架内部的原始工作方式，但它不是一个封闭的框。此外，如果您认为查看这些方法的源代码会显示明显的低效率和优化机会，那么我经常发现这并不容易发现，您最好阅读文档和测量。

Answer 2

我认为这比接受的答案简单得多，而且它根本不使用System.Linq命名空间。

public class Program
{
    public static void Main()
    {
        IList<IList<int>> lists = new List<IList<int>>
        {
            new List<int> {1, 2, 4, 4},
            new List<int> {3, 4, 5},
            new List<int> {4, 2, 1, 4},
            new List<int> {1, 2, 2},
            new List<int> {1, 2},
        };

        // There is no Multiset data structure in C#, but we can represent it as a set of tuples,
        // where each tuple contains an item and the number of its occurrences.

        // The dictionary below would not allow to add the same multisets twice, while keeping track of the original lists.
        var multisets = new Dictionary<HashSet<Tuple<int, int>>, IList<int>>(HashSet<Tuple<int, int>>.CreateSetComparer());
        foreach (var list in lists)
        {
            // Count the number of occurrences of each item in the list.
            var set = new Dictionary<int, int>();
            foreach (var item in list)
            {
                int occurrences;
                set[item] = set.TryGetValue(item, out occurrences) ? occurrences + 1 : 1;
            }

            // Create a set of tuples that we could compare.
            var multiset = new HashSet<Tuple<int, int>>();
            foreach (var kv in set)
            {
                multiset.Add(Tuple.Create(kv.Key, kv.Value));
            }

            if (!multisets.ContainsKey(multiset))
            {
                multisets.Add(multiset, list);
            }
        }

        // Print results.
        foreach (var list in multisets.Values)
        {
            Console.WriteLine(string.Join(", ", list));
        }
    }
}

输出将是：

1, 2, 4, 4
3, 4, 5
1, 2, 2
1, 2

删除没有Linq for IList IList的重复项

2 个答案: