模糊比较加权场(建议类似实例)

时间:2014-08-09 18:20:58

标签: c# fuzzy-search weighted search-suggestion

今天我遇到了一个特定的任务并且喜欢用干净的代码解决它,因此决定与其他同学分享它很酷 - 但是,嘿,让我们保持一个问题的格式

任务:

给定类型T(源)的实例和类型T的实例集合(可能的建议), 提供与源类似的建议,按相似性排序,并完全排除其相似性低于特定阈值的建议。

相似性将是实例的多个字段的模糊字符串比较,每个字段具有重要性权重。

示例输入:

来源实例:

{A = "Hello", B = "World", C = "and welcome!"}

可能的建议:

{A = "Hola", B = "World", C = "Welcome!"}
{A = "Bye", B = "world", C = "and fairwell"}
{A = "Hell", B = "World", C = "arrives..."}
{A = "Hello", B = "Earth", C = "and welcome!"}
{A = "Hi", B = "world", C = "welcome!"}

字段的重要性:

  • 答:30%
  • B:50%
  • C:20%

示例输出:

[0] = {A = "Hell", B = "World", C = "arrives..."}
[1] = {A = "Hola", B = "World", C = "Welcome!"}
[2] = {A = "Hello", B = "Earth", C = "and welcome!"}
[3] = {A = "Hi", B = "world", C = "welcome!"}

请注意,可能的建议Bye;world;and fairwell根本不存在,因为它没有达到最小相似度阈值(假设阈值至少为50%加权相似度)

第一个结果与来源最相似,即使C字段与来源完全不相似,因为我们为C提供了一个低至20%的权重,另外两个更重的加权字段与源非常相似(或完全匹配)。

模糊比较侧注

用于比较string astring b的算法可以是任何已知的模糊比较算法,这不是真正的重点。

那么如何将可能的建议列表转换为有序建议的实际列表呢?(哦,主,请帮忙等)

1 个答案:

答案 0 :(得分:0)

对于我们的情况,让我们使用真棒Levenshtein distance算法。

假设我们有一个具有以下签名的函数:

private static int CalcLevenshteinDistance(string a, string b)

要实际获得ab之间的相似性,而不是距离,我们将使用:

private static decimal CalcLevenshteinSimilarity(string a, string b)
{
    return 1 - ((decimal)CalcLevenshteinDistance(a, b) /
                Math.Max(a.Length, b.Length));
}

如果字符串完全相同,则返回1,如果字符串完全不相似,则返回0。例如,0.89 ab 89%相似(不错!)

为了帮助我们处理加权字段,让我们创建一个小帮助类:

public class SuggestionField
{
    public string SourceData { get; set; }
    public string SuggestedData { get; set; }
    public decimal Importance { get; set; }
}

这将代表将T类型的单个字段与源T实例匹配所需的所有信息。

现在计算单个建议与来源之间的加权相似性非常简单:

private static decimal RateSuggestion(IEnumerable<SuggestionField> fields)
{
    return fields.Sum(x =>
        x.Importance * CalcLevenshteinSimilarity(x.SourceData,
                                                 x.SuggestedData));
}

现在让我们将它包装在一个能够获得所有可能建议的函数中,以及SuggestionField以非常酷且易于使用的方式包装:

public static IEnumerable<T> Suggest<T>
    (IEnumerable<T> possibleSuggestions,
     params Func<T, SuggestionField>[] fieldSelectors)
{
    return possibleSuggestions
        .Select(x => new
                     {
                         Suggestion = x,
                         Similarity = RateSuggestion(fieldSelectors.Select(f => f(x)))
                     })
        .OrderByDescending(x => x.Similarity)
        .TakeWhile(x => x.Similarity > 0.5m) // <-- Threshold here!
        .Select(x => x.Suggestion);
}

好吧,好吧,乍看之下这段代码可能有点混乱,但放松一下。 主要的混淆可能来自params Func<T, SuggestionField>[] fieldSelectors,因此来自Similarity = RateSuggestion(fieldSelectors.Select(f => f(x)))

对于那些在Linq上有实力的人以及所有那些有选择器的游戏,人们可能已经理解了如何使用该功能。无论如何,只需片刻就可以了!

用法:

// I'll be using anonymous types here, but you don't have to be lazy about it
var src = new {A = "Hello", B = "World", C = "and welcome!"};
var possibleSuggestions =
    new[]
    {
        new {A = "Hola", B = "World", C = "Welcome!"},
        new {A = "Bye", B = "world", C = "and fairwell"},
        new {A = "Hell", B = "World", C = "arrives..."},
        new {A = "Hello", B = "Earth", C = "and welcome!"},
        new {A = "Hi", B = "world", C = "welcome!"}
    };

var suggestions =
    Suggest(possibleSuggestions,
            x => new SuggestionField
                 {
                     SourceData = src.A,
                     SuggestedData = x.A,
                     Importance = 0.3m // 30%
                 },
            x => new SuggestionField
                 {
                     SourceData = src.B,
                     SuggestedData = x.B,
                     Importance = 0.5m // 50%
                 },
            x => new SuggestionField
                 {
                     SourceData = src.C,
                     SuggestedData = x.C,
                     Importance = 0.2m // 20%
                 }).ToArray();

这对你来说可能看起来不错,或者可以根据自己的喜好进行更改,但我希望这个想法很明确,有人会发现它很有用;)

<强> P.S

当然,相似性阈值可以作为参数传递。 随意添加任何想法和评论如何使这更好或更可读!