卦比较

Question

我对编码很新，所以我想我自己没有看到明显的答案，所以如果这是一个愚蠢的问题，我很抱歉，但我真的被困在这里。我试图比较来自两个不同文本（A和B）的两组三元组。如果A中存在B中没有三元组，那么我会说这两个文本是不同的，至少对于我目前的目的而言。我使用Nuve进行三元组提取。

到目前为止，我有这个：

        var paragraph = "This is not a phrase. This is not a sentence.";
        var paragraph2 = "This is a phrase. This is a sentence. This have nothing to do with sentences.";
        ITokenizer tokenizer = new ClassicTokenizer(true);
        SentenceSegmenter segmenter = new TokenBasedSentenceSegmenter(tokenizer);
        var sentences = segmenter.GetSentences(paragraph);
        ITokenizer tokenizer2 = new ClassicTokenizer(true);
        SentenceSegmenter segmenter2 = new TokenBasedSentenceSegmenter(tokenizer2);
        var sentences2 = segmenter2.GetSentences(paragraph2);


        var extractor = new NGramExtractor(3);
        var grams1 = extractor.ExtractAsList(sentences);
        var grams2 = extractor.ExtractAsList(sentences2);

        var nonintersect = grams2.Except(grams1);


        foreach (var nGram in nonintersect)
        {
            var current = nGram;
            bool found = false;
            foreach (var n in grams2)
            {

                if (!found)
                {
                    if (n == current)
                    {
                        found = true;

                    }
                }
            }
            if (!found)
            {
                var result = current;
                string finalresult = Convert.ToString(result);
                textBox3.AppendText(finalresult+ "\n");

            }

这样我希望得到B中的句子不存在于A中（即示例中B中的所有句子），但现在我必须将B中的每个三元组与A中的每个三元组进行比较。看看句子是否真的彼此不同。我试图用另一个嵌套的foreach这样做，但我得到的是无意义的数据，如下所示：

 foreach (var sentence2 in sentences2)
        {
            var actual = sentence2;
            bool found1 = false;

            foreach (var sentence in sentences)
            {

                if (!found1) 
                {
                   if (actual == sentence)
                   {
                   found1 = true;
                   }
                }

            }

            if (!found1)
            {

                    string finalresult= Convert.ToString(actual);
                    textBox3.AppendText(finalresult+ "\n");

            }

        }

这样做我试着验证B中每个句子的三元组是否等于A中每个句子的三元组，如果是，则textBox3将为空。

简单地说，我正在尝试编写与Ferret类似的内容，但是对于C＃，仅用于比较两个给定的纯文本。据我所知，C＃没有类似的内容。

任何帮助或提示都会非常感激。谢谢！

Answer 1

比较文字正文

比较两个文本主体并将它们标记为相似，如果它们至少有一个共同的句子级三元组是非常简单的：

public bool AreTextsSimilar(string a, string b)
{
    // We can reuse these objects - they could be stored in member fields:
    ITokenizer tokenizer = new ClassicTokenizer(true);
    SentenceSegmenter segmenter = new TokenBasedSentenceSegmenter(tokenizer);
    NGramExtractor trigramExtractor = new NGramExtractor(3);

    IEnumerable<string> sentencesA = segmenter.GetSentences(a);
    IEnumerable<string> sentencesB = segmenter.GetSentences(b);

    // The order of trigrams doesn't matter, so we'll fetch them as sets instead,
    // to make comparisons between their elements more efficient:
    ISet<NGram> trigramsA = trigramExtractor.ExtractAsSet(sentencesA);
    ISet<NGram> trigramsB = trigramExtractor.ExtractAsSet(sentencesB);

    // 'Intersect' returns all elements that are found in both collections:
    IEnumerable<NGram> sharedTrigrams = trigramsA.Intersect(trigramsB);

    // 'Any' only returns true if the collection isn't empty:
    return sharedTrigrams.Any();
}

没有Linq方法（Intersect，Any），最后两行可以实现为循环：

    foreach (NGram trigramA in trigramsA)
    {
        // As soon as we find a shared sentence trigram we can conclude that
        // the two bodies of text are indeed similar:
        if (trigramsB.Contains(trigramA))
            return true;
    }
    return false;
}

没有共享字卦的句子

检索所有不共享单词级三元组的句子需要更多的工作：

public IEnumerable<string> GetUniqueBSentences(string a, string b)
{
    // We can reuse these objects - they could be stored in member fields:
    ITokenizer tokenizer = new ClassicTokenizer(true);
    SentenceSegmenter segmenter = new TokenBasedSentenceSegmenter(tokenizer);
    NGramExtractor trigramExtractor = new NGramExtractor(3);

    IEnumerable<string> sentencesA = segmenter.GetSentences(a);
    IEnumerable<string> sentencesB = segmenter.GetSentences(b);

    ITokenizer wordTokenizer = new ClassicTokenizer(false);
    foreach (string sentenceB in sentencesB)
    {
        IList<string> wordsB = wordTokenizer.Tokenize(sentenceB);
        ISet<NGram> wordTrigramsB = trigramExtractor.ExtractAsSet(wordsB);

        bool foundMatchingSentence = false;
        foreach (string sentenceA in sentencesA)
        {
            // This will be repeated for every sentence in B. It would be more efficient
            // to generate trigrams for all sentences in A once, before we enter these loops:
            IList<string> wordsA = wordTokenizer.Tokenize(sentenceA);
            ISet<NGram> wordTrigramsA = trigramExtractor.ExtractAsSet(wordsA);

            if (wordTrigramsA.Intersect(wordTrigramsB).Any())
            {
                // We found a sentence in A that shares word-trigrams, so stop comparing:
                foundMatchingSentence = true;
                break;
            }
        }

        // No matching sentence in A? Then this sentence is unique to B:
        if (!foundMatchingSentence)
            yield return sentenceB;
    }
}

显然segmenter还会返回一个额外的空句，您可能希望将其过滤掉（或弄清楚如何阻止segmenter这样做。）

我确定如果考虑到性能，可以优化上面的代码，但我会把这个留给你。