我的查询花了太多时间执行?

时间:2018-03-12 15:01:12

标签: mysql sql performance symfony symfony1

我从拥有数百万条记录的数据库中获取用户线索,并且提取时间为5-10秒。我怎样才能让它更快?我应该从ORM切换到直接查询还是可以使用当前的ORM执行某些操作?

public static function getBtqUsers($search_params) {
    $criteria = new Criteria();

    $criteria->clearSelectColumns();
    $criteria->addSelectColumn("btq_user.id as id");
    $criteria->addSelectColumn("btq_user.drid as drid");
    $criteria->addSelectColumn("btq_user.name as name");
    $criteria->addSelectColumn("btq_user.email as email");
    $criteria->addSelectColumn("btq_user.active as active");
    $criteria->addSelectColumn("btq_user.lead_from as lead_from");
    $criteria->addSelectColumn("btq_user.source as source");
    $criteria->addSelectColumn("btq_user.http_referer as http_referer");
    $criteria->addSelectColumn("btq_user.sms_status as sms_status");
    $criteria->addSelectColumn("btq_user.telefon as telefon");
    $criteria->addSelectColumn("btq_user.datain as datain");
    $criteria->addSelectColumn("btq_user.kall as kall");
    $criteria->addSelectColumn("btq_user.ip_address as ip_address");

    $criteria->addSelectColumn("count(btq_user_track_blog_video.counter) as total_video_viewed");

    $criteria->addSelectColumn("btq_doctors.id as dr_id");
    $criteria->addSelectColumn("btq_doctors.drname as dr_name");
    $criteria->addSelectColumn("btq_doctors.drphone as dr_phone");
    $criteria->addSelectColumn("btq_doctors.initials as dr_initials");
    $criteria->addSelectColumn("btq_doctors.drmail as dr_mail");

    $criteria->addSelectColumn("event_data.btq_user_id as event_user_id");
    $criteria->addSelectColumn("count(if(event_data.event_parent_id=2, event_data.event_parent_id, null)) as kms_total_video_viewed");
    $criteria->addSelectColumn("count(if(event_data.event_parent_id=1, event_data.event_parent_id, null)) as kms_total_blog_viewed");

    $criteria->addSelectColumn("state.pres as pres");
    $criteria->addSelectColumn("state.state as state");

    $criteria->addSelectColumn("btq_user_sales_choice.type as choice_type");
    $criteria->addSelectColumn("btq_user_sales_choice.opt_value as choice_value");
    $criteria->addSelectColumn("btq_user_sales_choice.opt_text as choice_text");
    $criteria->addSelectColumn("btq_user.pfu_customer_id as pfu_customer_id");
    $criteria->addSelectColumn("lead_schedule.id as schedule_id");
    $criteria->addSelectColumn("lead_schedule.created_at as schedule_date");

    $criteria->addJoin(self::STATE_ID, StatePeer::ID, Criteria::LEFT_JOIN);
    $criteria->addJoin(self::BTQ_USER_SALES_CHOICE_ID, BtqUserSalesChoicePeer::ID, Criteria::LEFT_JOIN);
    $criteria->addJoin(self::ID, LeadSchedulePeer::LEAD_ID, Criteria::LEFT_JOIN);
    $criteria->addJoin(self::DRID, BtqDoctorsPeer::ID, Criteria::LEFT_JOIN);
    $criteria->addJoin(self::ID, BtqUserTrackBlogVideoPeer::USER_ID, Criteria::LEFT_JOIN);
//    $criteria->addJoin(self::DRID, BtqDoctorPfuAdminPeer::BTQ_DOCTOR_ID, Criteria::LEFT_JOIN);
//    $criteria->addExtraJoin(BtqDoctorPfuAdminPeer::PFU_ADMIN_ID, PfuCustomerPeer::PFU_ADMIN_ID, Criteria::LEFT_JOIN, " AND " . PfuCustomerPeer::EMAIL . " = " . self::EMAIL);

    $criteria->addJoin(self::ID, EventDataPeer::BTQ_USER_ID, Criteria::LEFT_JOIN);

    //$criteria->addGroupByColumn(self::ID);
    $criteria->addGroupByColumn(self::EMAIL);
    //$criteria->addGroupByColumn(EventDataPeer::BTQ_USER_ID);
    $criteria->add(BtqUserPeer::IS_DUMMY_DETAIL, "1", Criteria::NOT_EQUAL);

    $criteria->addDescendingOrderByColumn(self::DATAIN);
    if (!empty($search_params)) {
      foreach ($search_params as $key => $param) {

        if (trim($param)) {
          $param = addslashes($param);

          switch ($key) {

            case 'name':
              $criteria->add(BtqUserPeer::NAME, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'email':
              $criteria->add(BtqUserPeer::EMAIL, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'dr_name':
              $criteria->add(BtqDoctorsPeer::DRNAME, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'phone':
              $criteria->add(BtqUserPeer::TELEFON, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'lead_from':
              $criteria->add(BtqUserPeer::LEAD_FROM, $param, Criteria::EQUAL);
              break;

            case 'start_date':
              $criteria->add(BtqUserPeer::DATAIN, $param . " 00:00:00", Criteria::GREATER_EQUAL);
              break;

            case 'end_date':
              $criteria->addAnd(BtqUserPeer::DATAIN, $param . " 23:59:59", Criteria::LESS_EQUAL);
              break;

            case 'location':
              if ($param == "Local")
                $criteria->add(StatePeer::PRES, array("MD", "VA", "DC"), Criteria::IN);
              else if ($param == "Non Local")
                $criteria->add(StatePeer::PRES, array("MD", "VA", "DC"), Criteria::NOT_IN);
              break;
          }
        }
      }
    }//FB::log($criteria);
    return $criteria;
//    return self::doSelectStmt($criteria)->fetchAll();
  }

我正在尝试取回2天的记录,但仍然需要花费太多时间。

代码我作为参数传递给searchParams:

> $searchParams = array('start_date' => date('Y-m-d', strtotime('-2
> days')) , 'end_date' => date("Y-m-d"));
>       $this->pager->setCriteria(btqAdminBtqUserPeer::getBtqUsers($searchParams));
>       $this->form->setDefault('start_date', $searchParams['start_date']);
>       $this->form->setDefault('end_date', $searchParams['end_date']);

RAW SQL:

SELECT  btq_user.id as id, btq_user.drid as drid, btq_user.name as name,
        btq_user.email as email, btq_user.active as active, btq_user.lead_from as lead_from,
        btq_user.source as source, btq_user.http_referer as http_referer,
        btq_user.sms_status as sms_status, btq_user.telefon as telefon,
        btq_user.datain as datain, btq_user.kall as kall, btq_user.ip_address as ip_address,
        count(btq_user_track_blog_video.counter) as total_video_viewed,
        btq_doctors.id as dr_id, btq_doctors.drname as dr_name,
        btq_doctors.drphone as dr_phone, btq_doctors.initials as dr_initials,
        btq_doctors.drmail as dr_mail, event_data.btq_user_id as event_user_id,
        count(if(event_data.event_parent_id=2, event_data.event_parent_id,
                        null)) as kms_total_video_viewed,
        count(if(event_data.event_parent_id=1,
                        event_data.event_parent_id, null)
             ) as kms_total_blog_viewed,
        state.pres as pres, state.state as state, btq_user_sales_choice.type as choice_type,
        btq_user_sales_choice.opt_value as choice_value, btq_user_sales_choice.opt_text as choice_text,
        btq_user.pfu_customer_id as pfu_customer_id, lead_schedule.id as schedule_id,
        lead_schedule.created_at as schedule_date
    FROM  btq_user
    LEFT JOIN  state  ON (btq_user.STATE_ID=state.ID)
    LEFT JOIN  btq_user_sales_choice
         ON (btq_user.BTQ_USER_SALES_CHOICE_ID=btq_user_sales_choice.ID)
    LEFT JOIN  lead_schedule  ON (btq_user.ID=lead_schedule.LEAD_ID)
    LEFT JOIN  btq_doctors  ON (btq_user.DRID=btq_doctors.ID)
    LEFT JOIN  btq_user_track_blog_video
         ON (btq_user.ID=btq_user_track_blog_video.USER_ID)
    LEFT JOIN  event_data  ON (btq_user.ID=event_data.BTQ_USER_ID)
    WHERE  btq_user.IS_DUMMY_DETAIL<>1
      AND  (btq_user.DATAIN>='2008-03-11 00:00:00'
      AND   btq_user.DATAIN<='2018-03-13 23:59:59'
           )
    GROUP BY  btq_user.EMAIL
    ORDER BY  btq_user.DATAIN DESC
    LIMIT  20

查看:

<?php
      foreach ($pager->getResults() as $row):
        $customer = btqAdminBtqUserPeer::getQuizUserGroupByEmaiIdAndTelephone($row['email'], $params);
        if ($customer['quizes']) {
          $quizzes = explode(",", $customer['quizes']);
          $quizzes_count = count($quizzes);
        } else {
          $quizzes = '';
          $quizzes_count = 0;
        }

        $scheduleClass = ($row['schedule_date']) ? 'schedule_available':'schedule_empty';
        ?>

从模板/视图我调用以下函数,这会使查询变慢:

public static function getQuizUserGroupByEmaiIdAndTelephone($userEmailId, $searchParameters) {
    $criteria = new Criteria();
    $criteria->addSelectColumn(" IF( COUNT(btq_user.id) > 1, GROUP_CONCAT(CONCAT(btq_user.id),'~',CONCAT(btq_user.datain)) , null) as quizes");
    $criteria->add(BtqUserPeer::EMAIL, $userEmailId, Criteria::EQUAL);
    $criteria->add(BtqUserPeer::IS_DUMMY_DETAIL, "1", Criteria::NOT_EQUAL);
    if (!empty($searchParameters)) {
      foreach ($searchParameters as $key => $param) {

        if (trim($param)) {
          $param = addslashes($param);

          switch ($key) {

            case 'name':
              $criteria->add(BtqUserPeer::NAME, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'email':
              $criteria->add(BtqUserPeer::EMAIL, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'dr_name':
              $criteria->add(BtqDoctorsPeer::DRNAME, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'phone':
              $criteria->add(BtqUserPeer::TELEFON, "%" . $param . "%", Criteria::LIKE);
              break;

            case 'lead_from':
              $criteria->add(BtqUserPeer::LEAD_FROM, $param, Criteria::EQUAL);
              break;

            case 'start_date':
              $criteria->add(BtqUserPeer::DATAIN, $param . " 00:00:00", Criteria::GREATER_EQUAL);
              break;

            case 'end_date':
              $criteria->addAnd(BtqUserPeer::DATAIN, $param . " 23:59:59", Criteria::LESS_EQUAL);
              break;

            case 'location':
              $criteria->addJoin(self::STATE_ID, StatePeer::ID, Criteria::LEFT_JOIN);
              if ($param == "Local")
                $criteria->add(StatePeer::PRES, array("MD", "VA", "DC"), Criteria::IN);
              else if ($param == "Non Local")
                $criteria->add(StatePeer::PRES, array("MD", "VA", "DC"), Criteria::NOT_IN);
              break;
          }
        }
      }
    }
$criteria->addDescendingOrderByColumn(self::DATAIN);
return self::doSelectStmt($criteria)->fetch(PDO::FETCH_ASSOC);

如果需要其他任何内容,请告诉我我将编辑问题和说明。

由于

1 个答案:

答案 0 :(得分:0)

如果您只是获得COUNT(*),则不需要所有列。 LEFT JOINs不影响计数(在本例中)。我想这会给你相同的数量:

        SELECT COUNT(DISTINCT btq_user.EMAIL)
            FROM  `btq_user`
            WHERE  btq_user.IS_DUMMY_DETAIL<>1
              AND  btq_user.DATAIN<='2018-02-28 23:59:59'

此索引可以帮助:

INDEX(DATAIN)

更多

这没有意义:

SELECT  IF( COUNT(btq_user.id) > 1,
                   GROUP_CONCAT(CONCAT(btq_user.id),
                                '~',CONCAT(btq_user.datain)) ,
                   null ) as quizes
    FROM  `btq_user`
    WHERE  btq_user.EMAIL='melinbakar@hotmail.com'
      AND  btq_user.IS_DUMMY_DETAIL<>1
      AND  (btq_user.DATAIN>='2008-03-11 00:00:00'
      AND   btq_user.DATAIN<='2018-03-13 23:59:59'
           )
    ORDER BY  btq_user.DATAIN DESC

只有一行(由于聚合函数COUNT),因此ORDER BY无法使用。也许你想要ORDER BY _inside_ the GROUP_CONCAT`?

那需要INDEX(email, datain)