如何在一个文本块中找出最常用的2个单词组合?

时间:2011-02-11 21:55:54

标签: php n-gram word-frequency

我怎样才能找出我之后使用的最常见的两个单词来自一个文本块?换句话说,有一个在线或离线(或代码)的工具,我可以复制和粘贴文本,它输出我最常用的两个字频率,如:

从最常用到最少:

“猫”2.9% “她说”1.8% “去了”1.2%

由于

2 个答案:

答案 0 :(得分:2)

  1. 将文本分成两个单词对(使用substrstrpos来帮助您)

    • 找到空格的第二个索引,使用strpos,然后在开始和第二个空格索引之间使用子串来获得两个单词对。
  2. 将每对添加到地图或集合(该对将是关键字)并设置值(如果它已存在于地图中,则增加值
  3. 解析完全文后,根据地图/集的大小和每对的值计算百分比。

答案 1 :(得分:1)

这很有意思,但是我有点意思,这应该让你开始,不应该是你的答案。

这基本上将单词分组为2,将它们编入数组并递增所找到的时间,最后转换为百分比:)

$data = 'In the first centuries of typesetting, quotations were distinguished merely by indicating the speaker, and this can still be seen in some editions of the Bible. During the Renaissance, quotations were distinguished by setting in a typeface contrasting with the main body text (often Italic type with roman, or the other way round). Block quotations were set this way at full size and full measure.
Quotation marks were first cut in type during the middle of the sixteenth century, and were used copiously by some printers by the seventeenth. In Baroque and Romantic-period books, they could be repeated at the beginning of every line of a long quotation. When this practice was abandoned, the empty margin remained, leaving an indented block quotation';

//Clean The Data from un required chars!
$data = preg_replace("/[^\w]/"," ",$data);

$segments = explode(" ",$data);
$indexes = array();

for($i=0;$i<count($segments);$i++)
{
   if($i == 0)
   {
      continue;
   }

   if(trim($segments[$i - 1]) != "" && trim($segments[$i]) != "")
   {
      $key = trim($segments[$i - 1]) . " " . trim($segments[$i]);
      if(array_key_exists($key,$indexes))
      {
          $indexes[$key]["count"]++;
      }else
      {
          $indexes[$key] = array(
              'count' => 1,
              'words' => $key
          );
      }
   }
}

//Change to the percentage:
$total_double_words = count($segments);
foreach($indexes as $id => $set)
{
    $indexes[$id]['percentage'] = number_format((($set['count']/ $total_double_words) * 100),2) . "%";
}

var_dump($indexes);

你可以在这里看到它:http://codepad.org/rcwpddW8