Question

我正在使用Yii createCommand QueryBuilder并构建下面的插入查询，下面的代码就像魅力（在3秒内插入6000条记录）但过了一段时间它的速度变得非常慢。

我的代码：

$taskmodel = TaskModel::model()->findAll($cri); // filtering using some criteria
$now = new DateTime();

foreach( $taskmodel as $tsk ) {
    $recall_date = $tsk['recall_date'] == "" ? "null" : '"'.$tsk['recall_date'].'"';
    $recall_agent = $tsk['recall_agent_id'] == "" ? "null" : $tsk['recall_agent_id'];
    $next_date = $tsk['next_action_date'] == "" ? "null" : '"'.$tsk['next_action_date'].'"';
    $pc_color = $tsk['postcode_color'] == "" ? "null" : '"'.$tsk['postcode_color'].'"';
    $cpc_id = $tsk['crm_campaign_post_code_id'] == "" ? "null" : $tsk['crm_campaign_post_code_id'];
    $priority = $tsk['priority'] == "" ? 10 : $tsk['priority'];                     
    $field1 = "null" ;
    $field2 = "null" 
    $field3 = "null" 
    $field4 = "null" ;
    $contact_timezone = $tsk['contact_timezone'] == "" ? "''" : '"'.$tsk['contact_timezone'].'"';

    $sql[] = '('.$tsk['crm_task_id'].', '.$tsk['crm_campaign_id'].', "'.$tsk['crm_contact_id'].'", "'.$now->format('Y-m-d H:i:s').'", "'.$now->format('Y-m-d H:i:s').'
                ", '.$tsk['crm_filter_id'].', '.$tsk['current_status'].', '.$priority.', '.$recall_date.', '.$recall_agent.', '.$next_date.
                ', '.$pc_color.', '.$cpc_id.', '.$sort.', '.$field1.', '.$field2.', '.$field3.', '.$field4.', '.$contact_timezone.', '.$tsk['is_active'].')';
    }

   if(sizeof($sql) > 0){
     $ins ='INSERT INTO crm_pending_task (crm_task_id,   crm_campaign_id,crm_contact_id,created,updated,crm_filter_id,    
                 current_status,priority,recall_date,recall_agent_id,next_action_date,postcode_color,crm_campaign_post_code_id,sort,
              field1,field2,field3,field4,contact_timezone,is_active) VALUES '.implode(',', $sql);
          Yii::app()->db->createCommand($ins)->execute();
                }

我发现一些有趣的东西，插入50000条记录后变得很慢!!!为什么会这样？

如何提高插入查询速度，直到完成所有插入？

Answer 1

我认为是因为你的请求占用了太多内存并且你正在交换。

要释放一些记忆，你应该尝试启用Gc并在你的foreach中启动它（可能不会在每次迭代时）

    gc_enable();
    gc_collect_cycles();

Answer 2

也许尝试将较小的块插入数据库。

foreach( $taskmodel as $tsk ) {
    $recall_date = $tsk['recall_date'] == "" ? "null" : '"'.$tsk['recall_date'].'"';
    $recall_agent = $tsk['recall_agent_id'] == "" ? "null" : $tsk['recall_agent_id'];
    $next_date = $tsk['next_action_date'] == "" ? "null" : '"'.$tsk['next_action_date'].'"';
    $pc_color = $tsk['postcode_color'] == "" ? "null" : '"'.$tsk['postcode_color'].'"';
    $cpc_id = $tsk['crm_campaign_post_code_id'] == "" ? "null" : $tsk['crm_campaign_post_code_id'];
    $priority = $tsk['priority'] == "" ? 10 : $tsk['priority'];                     
    $field1 = "null" ;
    $field2 = "null" 
    $field3 = "null" 
    $field4 = "null" ;
    $contact_timezone = $tsk['contact_timezone'] == "" ? "''" : '"'.$tsk['contact_timezone'].'"';

    $sql[] = '('.$tsk['crm_task_id'].', '.$tsk['crm_campaign_id'].', "'.$tsk['crm_contact_id'].'", "'.$now->format('Y-m-d H:i:s').'", "'.$now->format('Y-m-d H:i:s').'
                ", '.$tsk['crm_filter_id'].', '.$tsk['current_status'].', '.$priority.', '.$recall_date.', '.$recall_agent.', '.$next_date.
                ', '.$pc_color.', '.$cpc_id.', '.$sort.', '.$field1.', '.$field2.', '.$field3.', '.$field4.', '.$contact_timezone.', '.$tsk['is_active'].')';
}


    $recordCount = sizeof($sql);
    $sliceSize = 250;

    for($x = 0; $x < $recordCount; $x += $sliceSize) {

        $ins = sprintf('
            INSERT INTO
                crm_pending_task 
                    (
                        crm_task_id, crm_campaign_id, crm_contact_id, created,updated, 
                        crm_filter_id, current_status, priority, recall_date, recall_agent_id,
                        next_action_date, postcode_color, crm_campaign_post_code_id, sort,
                        field1, field2, field3, field4, contact_timezone, is_active
                    )
            VALUES
                %s; ', implode(',', array_slice($sql, $x, $sliceSize));

        Yii::app()->db->createCommand($ins)->execute();
    }

}

MySQL插入查询进展缓慢

2 个答案: