MySQL + PHP数百万行,从多个查询中排序数据。记忆消耗

时间:2014-09-03 15:39:21

标签: php mysql sorting

所以我有2张桌子。一个表包含所有实际的campaign_id信息。 第二个表格包含有关campaign_id

的展示/统计信息

我在页面上有一个表格(我使用ajax,但除此之外)。我想“排序”一列,但所有行都是由campaign_id表生成的,我首先运行每个广告系列的所有统计信息,然后将它们链接到每一行。然后在所有信息/数据都启动后,它会对所有信息/数据进行排序。这使用了大量的内存和资源。 这有效吗?是否有更好的解决方案来排序大量数据?

// I have to increase memory because the sorting takes a lot of resource
ini_set('memory_limit','1028M');

// the column I want to sort
$sortcolumn = $this->input->post('sortcolumn');
// direction of the sort ASC/DESC
$sortby = $this->input->post('sortby');

// millions of impression data that is linked with campaign_id
$cdata= array();
$s  = "SELECT report_campaign_id,";
$s .= "SUM(report_imps) AS imps ";
$s .= "FROM Campaign_Impressions"; 
$s .= "GROUP BY report_campaign_id ";
$r = $this->Qcache->result($s,0,'campaignsql');
foreach($r as $c) {

  $cdata[$c->report_campaign_id]['imps'] = ($c->imps) ? $c->imps : 0;
}


// 500,000+ thousand campaigns
// I draw my table from these campaigns
$rows = array();
$s = "SELECT * FROM Campaigns ";
$r = $this->db->query($s)->result();
foreach($r as $c)
{
   $row= array();
   $row['campaign_id'] = $c->campaign_id;
   // other campaign info here...

   // campaign statistics here...
   $row['campaign_imps'] = $cdata[$c->campaign_id]['imps'];

   // table row
   $rows[] = $row;
}


// prepare the columns i want to sort
$sortc = array();
foreach($rows as $sortarray) {
  if (!isset($sortarray[ $sortcolumn ])) continue;
  $sortc[] = str_replace(array('$',','),'',$sortarray[ $sortcolumn ]);
}

// sort columns and direction 
array_multisort($sortc,(($sortby==='asc')?SORT_ASC:SORT_DESC),SORT_NATURAL,$rows);

正如您所看到的,“campaign_impressions”表正在运行“每个”广告系列的数据,并且看起来效率不高,但更有效,而不是每行运行查询以了解数据。

(我不显示所有广告系列,但我需要运行其中每个广告系列以了解所有广告系列的排序)

1 个答案:

答案 0 :(得分:0)

您应该让MySQL使用order by

完成工作

如果在MySQL端仍需要花费大量时间,请考虑在列上使用排序索引