MySQL优化:单个查询中的多种模式(最常见的值)

时间:2019-03-19 17:08:45

标签: mysql optimization statistics analytics

我的房地产网站的一项功能允许用户订阅特定市场并通过电子邮件接收定期更新(称为“市场分析”)。分析需要将几个值作为模式(最常见的值)进行计算。经过研究,我了解到MySQL does not have a MODE() function特别是因为可能存在多种模式,而且可能根本没有模式,而且还因为在该列中至少包含两个值,您甚至无法获得单一模式。 / p>

  

哪个引导我访问此查询

SELECT AVG(Price) as AveragePrice,
AVG(BedroomsTotal) as AverageNumberOfBedrooms,
AVG(BathroomsTotal) as AverageNumberOfBathrooms,
AVG(SquareFeetTotal) as AverageSquareFeetTotal,
AVG(LotSize) as AverageLotSize,
AVG(AssociationFee) as AverageAssociationFee,
(SELECT PropertyType FROM (SELECT PropertyType, count(PropertyType) AS magnitude
FROM listings
GROUP BY PropertyType
ORDER BY magnitude DESC
LIMIT 1) as mpt) as MajorityPropertyType,
(SELECT magnitude FROM (SELECT PropertyType, count(PropertyType) AS magnitude
FROM listings
GROUP BY PropertyType
ORDER BY magnitude DESC
LIMIT 1) as mptc) as MajorityPropertyTypeCount,
(SELECT ArchitecturalStyle FROM (SELECT ArchitecturalStyle, count(ArchitecturalStyle) AS magnitude
FROM listings
GROUP BY ArchitecturalStyle
ORDER BY magnitude DESC
LIMIT 1) as mas) as MajorityArchitecturalStyle,
(SELECT magnitude FROM (SELECT ArchitecturalStyle, count(ArchitecturalStyle) AS magnitude
FROM listings
GROUP BY ArchitecturalStyle
ORDER BY magnitude DESC
LIMIT 1) as masc) as MajorityArchitecturalStyleCount,
AVG(YearBuilt) as AverageYearBuilt,
(SELECT PropertyCondition FROM (SELECT PropertyCondition, count(PropertyCondition) AS magnitude
FROM listings
GROUP BY PropertyCondition
ORDER BY magnitude DESC
LIMIT 1) as mpc) as MajorityPropertyCondition,
(SELECT magnitude FROM (SELECT PropertyCondition, count(PropertyCondition) AS magnitude
FROM listings
GROUP BY PropertyCondition
ORDER BY magnitude DESC
LIMIT 1) as mpcc) as MajorityPropertyConditionCount
FROM srep.active_listings 
WHERE concat(City, ', ', StateOrProvince)
LIKE "Boston, MA";

该查询工作正常,但是问题在于执行该查询需要花费10秒钟的时间,而查询成本为11,000,甚至不包含应放在WHERE子句中的条件语句的一小部分。还需要包含其他18条条件语句。

问题:

  
    

如何优化此查询?我应该使用较新版本的MySQL吗?我是否应该完全使用其他数据库?

  

当前执行计划

Current Execution Plan

结果

enter image description here

2 个答案:

答案 0 :(得分:1)

一种改进是减少了一半的子查询:

package org.ccf.flutter.plugin.mychart_plugin

import io.flutter.plugin.common.MethodCall
import io.flutter.plugin.common.MethodChannel
import io.flutter.plugin.common.MethodChannel.MethodCallHandler
import io.flutter.plugin.common.MethodChannel.Result
import io.flutter.plugin.common.PluginRegistry.Registrar

import epic.mychart.android.library.api.authentication.WPAPIAuthentication;
import epic.mychart.android.library.api.general.WPAccessResult;
import epic.mychart.android.library.api.prelogin.WPAPIPrelogin;
import epic.mychart.android.library.api.springboard.WPAPISpringboard;
import android.widget.Toast
import android.util.Log
import android.content.Intent
import android.app.Activity
import android.support.v4.app.FragmentManager
import android.content.Context
import io.flutter.plugin.common.PluginRegistry

class MychartPlugin: MethodCallHandler, WPAPIAuthentication.IWPOnLoginListener {
  private var registrar: PluginRegistry.Registrar? = null
  private val LOGIN_REQUEST_CODE = 9876

  fun MychartPlugin(registrar: PluginRegistry.Registrar) {
    this.registrar = registrar
  }
  companion object {
    @JvmStatic
    fun registerWith(registrar: Registrar) {
      val channel = MethodChannel(registrar.messenger(), "mychart_plugin")
      channel.setMethodCallHandler(MychartPlugin())
    }
  }

  override fun onMethodCall(call: MethodCall, result: Result) {
    if (call.method == "getPlatformVersion") {
      result.success("Android ${android.os.Build.VERSION.RELEASE}")
    } else if (call.method == "MyChartSdkLogin") {
      WPAPIAuthentication.login(this, "TURKJ123", "TurkJ123", LOGIN_REQUEST_CODE)
      result.success("called MyChartSdkLogin")
    }  else {
      result.notImplemented()
    }
  }


  override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent) {
    // super.onActivityResult(requestCode, resultCode, data)
    if (requestCode == LOGIN_REQUEST_CODE) {
      if (resultCode == Activity.RESULT_OK) {
        // login successful
        Log.i("LoginFragment", "OK")

      } else {
        // login failed
        val result = WPAPIAuthentication.getLoginResult(data)
        Log.i("LoginFragment", result.toString())
        val errorMessage = result.getErrorMessage(getContext())
        if (!errorMessage.isEmpty()) {
          Toast.makeText(getContext(), errorMessage, Toast.LENGTH_LONG).show()
        }
      }
    }
  }

  override fun startActivityForResult(p0: Intent, p1: Int) {
    // super.startActivityForResult(p0, p1)
  }

  override fun getSupportFragmentManager(): FragmentManager {
    return this.registrar.getSupportFragmentManager()
  }

  override fun getContext(): Context {
    return registrar!!.context()
  }
}

->

    (   SELECT  PropertyType
            FROM  
            (
                SELECT  PropertyType, count(PropertyType) AS magnitude
                    FROM  listings
                    GROUP BY  PropertyType
                    ORDER BY  magnitude DESC
                    LIMIT  1) as mpt
    ) as MajorityPropertyType,

此特定查询将需要 ( SELECT PropertyType FROM listings GROUP BY PropertyType ORDER BY COUNT(*) DESC LIMIT 1 ) as MajorityPropertyType (除非它已经是INDEX(PropertyType))。

另一个改进是避免在函数调用中隐藏索引列:

PRIMARY KEY

->

WHERE concat(City, ', ', StateOrProvince) LIKE "Boston, MA"

与复合词WHERE City = 'Boston' AND StateOrProvince = 'MA' 一起(以任意顺序)。这样可以避免扫描整个表,而只查看波士顿MA行。

即使有一个INDEX(City, StateOrProvince)函数,它也可能没有更快的速度-它实际上必须执行代码所要做的事情。

答案 1 :(得分:0)

使用另一种语言(例如Python)来获取列模式。这是使用Web API的示例。您必须先安装mysqlclientflask软件包,然后此代码才能起作用。

App.py

import MySQLdb
import MySQLdb.cursors
from statistics import mode
from flask import Flask, jsonify

app = Flask(__name__)

RES = {}

@app.route('/')
def bar():
  conn = MySQLdb.connect('localhost', user='root', cursorclass=MySQLdb.cursors.DictCursor)
  cursor = conn.cursor()
  sql = 'SELECT ArchitecturalStyle FROM srep.active_listings'
  cursor.execute(sql)
  data = cursor.fetchall()
  row = [obj['ArchitecturalStyle'] for obj in data]
  RES["ArchitecturalStyle"] = mode(row)
  return jsonify(RES)

  

使用所有10个属性进行测试

Python Web API using Flask

如您所见,在Python中(与MySQL相比),它花费了1/10的时间来获得相同的结果。