PHP - 将多维子数组相互比较并合并相似性阈值

时间:2017-03-05 15:27:31

标签: php arrays multidimensional-array compare levenshtein-distance

简介 - 此问题已于2018年5月27日更新:

我有1个 PHP 多维数组,包含6个子数组,每个子数组包含20个子子数组,每个子数组又包含2个子子数组,一个是字符串(标题),另一个是未指定数量的关键字(关键字)。

我希望将120个子子阵列中的每一个与Stillint 5子阵列中包含的100个其他子子阵列进行比较。使子阵列 1 中的子子阵列 1 子阵列进行比较在子阵列 2 1 并包括子阵列 20 包括子数组 6 ,等等。

如果两个子子阵列中的 关键字 被认为相同且 标题 也是如此,两者都使用Levenshtein距离,子子阵列将被合并。

示例脚本

我已经编写了一个完成此操作的脚本,但是对于两个独立的数组来说明我的目标:

<?php
// Variable deciding maximum Levenshtein distance between two words. Can be changed to lower / increase threshhold for whether two keywords are deemed identical.
$lev_point_value = 3;

// Variable deciding minimum amount of identical (passed the $lev_point_value variable) keywords needed to merge arrays. Can be changed to lower / increase threshhold for how many keywords two arrays must have in common to be merged.
$merge_tag_value = 4;

// Variable deciding minimum Levenshtein distance between two headers needed to merge arrays. Can be changed to lower / increase threshhold for whether two titles are deemed identical.
$merge_head_value = 22;

// Array1 - A story about a monkey, includes at header and keywords.
$array1 = array (
        "header" => "This is a story about a monkey.",
        'keywords' => array( "Trees", "Monkey", "Flying", "Drink", "Vacation", "Coconut", "Big", "Bonobo", "Climbing"
    ));

// Array1 - Another, but slightly different story about a monkey, includes at header and keywords.
$array2 = array (
        "header" => "This is another, but different story, about a monkey.",
        'keywords' => array( "Monkey", "Big", "Trees", "Bonobo", "Fun", "Dance", "Cow", "Coconuts"
    ));

// Function comparing keywords between two arrays. Uses levenshtein distance lesser than $lev_point_value. Each pass increases $merged_tag, which is then returned.
function sim_tag_index($array1, $array2, $lev_point_value) {
    $merged_tag = 0;
    foreach ($array1['keywords'] as $item1){
        foreach ($array2["keywords"] as $item2){
            if (levenshtein($item1, $item2) <= $lev_point_value) {
            $merged_tag++;
            };
         }
    };
    return $merged_tag;
}

// Function comparing headers between two arrays using levenshtein distance, which is then returned as $merged_head.
function sim_header_index($array1, $array2) {
    $merged_head = (levenshtein($array1['header'], $array2['header']));
    return $merged_head;
}

// Function running sim_tag_index against $merge_tag_value, if it passes, then running sim_tag_index against $merge_head_value, if this passes aswell, merge arrays.
function merge_on_sim($array1, $array2, $merge_tag_value, $merge_head_value, $lev_point_value) {
    $group = array();
    if (sim_tag_index($array1, $array2, $lev_point_value) >= $merge_tag_value) {
        if (sim_header_index($array1, $array2) >= $merge_head_value) {
            $group = (array_unique(array_merge($array1["keywords"],$array2["keywords"])));
        }
    }
    return $group;
}

// Printing function merge_on_sim.
print_r (merge_on_sim($array1, $array2, $merge_tag_value, $merge_head_value, $lev_point_value));
?>

问题:

如何扩展或重写我的脚本以通过多个子子阵列,将它们与其他子阵列中的所有其他子子阵列进行比较,然后合并被视为的子子阵列完全相同?

多维数组结构

$array = array (
    // Sub-array 1
    array (
        // Story 'Monkey 1' - Has identical sub-sub-arrays 'Monkey 2' and 'Monkey 3' and will be merged with them.
        array (
            "header" => "This is a story about a monkey.",
            'keywords' => array( "Trees", "Monkey", "Flying", "Drink", "Vacation", "Coconut", "Big", "Bonobo", "Climbing")
        ),
        // Story 'Cat 1' - Has identical sub-sub-array 'Cat 2' and will be merged with it.
        array (
            "header" => "Here's a catarific story about a cat",
            'keywords' => array( "meauw", "raaaw", "kitty", "growup", "Fun", "claws", "fish", "salmon")
        )
    ),
    // Sub-array 2
    array ( 
        // Story 'Monkey 2' - Has identical sub-sub-arrays 'Monkey 1' and 'Monkey 3' and will be merged with them.
        array (
            "header" => "This is another, but different story, about a monkey.",
            'keywords' => array( "Monkey", "Big", "Trees", "Bonobo", "Fun", "Dance", "Cow", "Coconuts")
        ),
        // Story 'Cat 2' - Has identical sub-sub-array 'Cat 1' and will be merged with it.
        array (
            "header" => "Here's a different story about a cat",
            'keywords' => array( "meauwe", "ball", "cat", "kitten", "claws", "sleep", "fish", "purr")
        )
    ),
    // Sub-array 3
    array ( 
        // Story 'Monkey 3' - Has identical sub-sub-arrays 'Monkey 1' and 'Monkey 2' and will be merged with them.
        array (
            "header" => "This is a third story about a monkey.",
            'keywords' => array( "Jungle", "tree", "monkey", "Bonobo", "Fun", "Dance", "climbing", "Coconut", "pretty")
        ),
        // Story 'Fireman 1' - Has no identical sub-sub-arrays and will not be merged.
        array (
            "header" => "This is a story about a fireman",
            'keywords' => array( "fire", "explosion", "burning", "rescue", "happy", "help", "water", "car")
        )
    )
);

通缉多维数组

$array = array (
    // Story 'Monkey 1', 'Monkey 2' and 'Monkey 3' merged.
    array (
        "header" => array( "This is a story about a monkey.", "This is another, but different story, about a monkey.", "This is a third story about a monkey."),
        'keywords' => array( "Trees", "Monkey", "Flying", "Drink", "Vacation", "Coconut", "Big", "Bonobo", "Climbing", "Fun", "Dance", "Cow", "Coconuts", "Jungle", "tree", "pretty")
    ),
    // Story 'Cat 1' and 'Cat 2' merged.
    array (
        "header" => array( "Here's a catarific story about a cat", "Here's a different story about a cat"),
        'keywords' => array( "meauw", "raaaw", "kitty", "growup", "Fun", "claws", "fish", "salmon", "ball", "cat", "kitten", "sleep", "fish", "purr")
    )
);

2 个答案:

答案 0 :(得分:1)

据我了解你的情况,你只有一个多维数组。第一级有6个数组,每个第一级数组包含20个(第二级)数组。每个二级数组包含1个(第三级)数组,其中包含2个元素:stringkeywords

您希望将每个stringkeywords与多维数组中存在的所有其他stringkeywords进行比较。

让我们用array_diff_key()&amp;做一些深思熟虑的循环。 range()

foreach($L1arrays as $index=>$L1focus){  // 6 arrays with indexes 0 - 5
    // now we want to check against all siblings that haven't been checked yet
    foreach(array_diff_key($L1arrays,range(0,$index)) as $L1comparison){  // no dupe loops
       // When $index=0, second foreach will only loop through $L1arrays indexed 1-5
       // When $index=1, second foreach will only loop through $L1arrays indexed 2-5
       // When $index=2, second foreach will only loop through $L1arrays indexed 3-5
       // When $index=3, second foreach will only loop through $L1arrays indexed 4-5
       // When $index=4, second foreach will only loop through $L1arrays[5]
       // When $index=5, it will not loop because it has already been compared
       //                to all other indexes via previous loops.

       /*
          ... in this section you can loop through the 20 sub-arrays of $L1focus
              and compare them against the 20 sub-arrays of each $L1comparison
       */
    }
}

现在讨论比较逻辑。我不能从您问题中的当前信息中了解您希望如何区分一个阵列是否与另一个阵列足够相似。我希望你会更加努力地创建一个真实的Wanted Multidimensional Array - 第二个数组的重复键为0,并且其中有两个空元素,所以它没有用。

您希望比较keywordsstring值,还是两者都比较?比较有多严格(&#39;相似性阈值&#39;)?

PHP是这类工作的绝佳表现者。我已从您的问题中删除了javascript标记。

请做一些工作/研究(利用我的循环建议)并精确完善您希望如何比较最深的数组值并编辑您的问题。要再次引起我对此问题的注意,请撰写以@mickmackusa

开头的评论 祝你好运!

答案 1 :(得分:1)

我会试一试!

我使用preg_grep查找其他子数组中相同的项目。然后我使用count来查看有多少匹配的关键字 这就是门槛。目前我将其设置为2,这意味着两个匹配的关键字匹配。

//flatten array to make it simpler
$new =[];
foreach($array as $subarr){
    $new = array_merge($new, $subarr);
}

$threshold = 2;
$merged=[];
foreach($new as $key => $story){
    // create regex pattern to find similar items
    $words = "/" . implode("|", $story["keywords"]) . "/i";
    foreach($new as $key2 => $story2){
        // only loop new items and items that has not been merged already
        if($key != $key2 && $key2 > $key && !in_array($key2, $merged)){
            // If the count of words from preg_grep is above threshold it's mergable
            if(count(preg_grep($words, $story2["keywords"])) > $threshold){
                // debug
                //echo $key . " " . $key2 . "\n";
                //echo $story["header"] . " = " . $story2["header"] ."\n\n";

                // if the item does not exist create it first to remove notices
                if(!isset($res[$key])) $res[$key] = ["header" => [], "keywords" =>[]];

                // add headers
                $res[$key]["header"][] = $story["header"];
                $res[$key]["header"][] = $story2["header"];

                // only keep unique 
                $res[$key]["header"] = array_unique($res[$key]["header"]);

                // add keywords and remove duplicates
                $res[$key]["keywords"] = array_merge($res[$key]["keywords"], $story["keywords"], $story2["keywords"]);
                $res[$key]["keywords"] = array_unique($res[$key]["keywords"]);

                // add key2 to merged so that we don't merge this again.
                $merged[] = $key2;
            }
        }
    }
}

var_dump($res);

https://3v4l.org/6cKRq

输出就像你想要的那样&#34;问题。