我有一组高度关系的表格,我已经将其展平为两个InnoDB表格:
mysql> desc WPropertyCube; +--------------------+---------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------------------+---------------------+------+-----+---------+----------------+ | id | bigint(20) unsigned | NO | PRI | NULL | auto_increment | | lineOfBusinessId | bigint(20) unsigned | NO | MUL | NULL | | | txtProperty1 | varchar(125) | YES | | NULL | | | txtProperty2 | varchar(125) | YES | | NULL | | | txtProperty3 | varchar(125) | YES | | NULL | | ... | txtProperty20 | varchar(125) | YES | | NULL | | | lookupPropertyId1 | bigint(20) unsigned | YES | | NULL | | | lookupPropertyId2 | bigint(20) unsigned | YES | | NULL | | | lookupPropertyId3 | bigint(20) unsigned | YES | | NULL | | ... | lookupPropertyId30 | bigint(20) unsigned | YES | | NULL | | +--------------------+---------------------+------+-----+---------+----------------+ mysql> show indexes from WPropertyCube; +---------------+------------+---------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | +---------------+------------+---------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | WPropertyCube | 0 | PRIMARY | 1 | id | A | 379383 | NULL | NULL | | BTREE | | | | WPropertyCube | 1 | WPropertyCube_idx01 | 1 | lineOfBusinessId | A | 204 | NULL | NULL | | BTREE | | | +---------------+------------+---------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> desc WMeasureCube; +----------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+---------------------+------+-----+---------+-------+ | propertyCubeId | bigint(20) unsigned | NO | PRI | NULL | | | actionDate | datetime | NO | PRI | NULL | | | measure1 | decimal(15,6) | YES | | NULL | | | measure2 | decimal(15,6) | YES | | NULL | | | measure3 | decimal(15,6) | YES | | NULL | | ... | measure30 | decimal(15,6) | YES | | NULL | | +----------------+---------------------+------+-----+---------+-------+ mysql> show indexes from WMeasureCube; +--------------+------------+-------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | +--------------+------------+-------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | WMeasureCube | 0 | PRIMARY | 1 | propertyCubeId | A | 18 | NULL | NULL | | BTREE | | | | WMeasureCube | 0 | PRIMARY | 2 | actionDate | A | 81680372 | NULL | NULL | | BTREE | | | | WMeasureCube | 1 | WMeasureCube_idx1 | 1 | actionDate | A | 18 | NULL | NULL | | BTREE | | | | WMeasureCube | 1 | WMeasureCube_idx1 | 2 | propertyCubeId | A | 81680372 | NULL | NULL | | BTREE | | | +--------------+------------+-------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
你可以看到我一直在玩索引,但还没有找到神奇的组合。 WPropertyCube
有~100K记录,WMeasureCube
有~80M。我的疑问是:
mysql> explain SELECT wmc.actionDate, -> SUM(wmc.measure7), -> SUM(wmc.measure8), -> SUM(wmc.measure9) -> FROM WMeasureCube wmc -> INNER JOIN WPropertyCube wpc -> ON wmc.propertyCubeId = wpc.id -> WHERE wpc.lineOfBusinessId IN ( 1, 2, 3, 4 ) -> AND wmc.actionDate BETWEEN '2010-06-28' AND '2010-09-26' -> GROUP BY wmc.actionDate -> ORDER BY wmc.actionDate; +----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+----------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+----------------------------------------------+ | 1 | SIMPLE | wmc | ALL | PRIMARY,WMeasureCube_idx1 | NULL | NULL | NULL | 81680372 | Using where; Using temporary; Using filesort | | 1 | SIMPLE | wpc | eq_ref | PRIMARY,WPropertyCube_idx01 | PRIMARY | 8 | db.wmc.propertyCubeId | 1 | Using where | +----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+----------------------------------------------+
此查询将~30K记录聚合到~1,000条记录中。请注意,没有索引用于访问WMeasureCube
,更糟糕的是,有一个文件流是slooooowww。查询时间范围在30到150秒之间。令人困惑的是,如果我删除了GROUP BY
:
mysql> explain SELECT wmc.actionDate, -> SUM(wmc.measure7), -> SUM(wmc.measure8), -> SUM(wmc.measure9) -> FROM WMeasureCube wmc -> INNER JOIN WPropertyCube wpc -> ON wmc.propertyCubeId = wpc.id -> WHERE wpc.lineOfBusinessId IN ( 1, 2, 3, 4 ) -> AND wmc.actionDate BETWEEN '2010-06-28' AND '2010-09-26' -> ORDER BY wmc.actionDate; +----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+-------------+ | 1 | SIMPLE | wmc | ALL | PRIMARY,WMeasureCube_idx1 | NULL | NULL | NULL | 81680372 | Using where | | 1 | SIMPLE | wpc | eq_ref | PRIMARY,WPropertyCube_idx01 | PRIMARY | 8 | db.wmc.propertyCubeId | 1 | Using where | +----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+-------------+
仍然没有索引使用,但至少没有文件输入。此查询在不到一秒的时间内始终运行。特别奇怪的是,在执行时间方面,按一组列排序应该是一致的,以便按同一组进行分组。
我该怎么做才能加快查询速度?
我试图添加所有服务器变量,但它们占用了太多空间。所以,我添加了我认为最有帮助的那些。
mysql> show variables; +---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------+ | Variable_name | Value | +---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------+ | big_tables | OFF | | binlog_cache_size | 4194304 | | binlog_direct_non_transactional_updates | OFF | | binlog_format | STATEMENT | | bulk_insert_buffer_size | 8388608 | | character_set_database | utf8 | | character_set_filesystem | binary | | default_storage_engine | InnoDB | | innodb_adaptive_flushing | ON | | innodb_adaptive_hash_index | ON | | innodb_additional_mem_pool_size | 16777216 | | innodb_autoextend_increment | 8 | | innodb_autoinc_lock_mode | 1 | | innodb_buffer_pool_instances | 1 | | innodb_buffer_pool_size | 6442450944 | | innodb_change_buffering | all | | innodb_checksums | ON | | innodb_commit_concurrency | 0 | | innodb_concurrency_tickets | 500 | | innodb_data_file_path | ibdata1:10M:autoextend | | innodb_data_home_dir | | | innodb_doublewrite | ON | | innodb_fast_shutdown | 1 | | innodb_file_format | Barracuda | | innodb_file_format_check | ON | | innodb_file_format_max | Barracuda | | innodb_file_per_table | ON | | innodb_flush_log_at_trx_commit | 0 | | innodb_flush_method | | | innodb_force_recovery | 0 | | innodb_io_capacity | 200 | | innodb_lock_wait_timeout | 120 | | innodb_locks_unsafe_for_binlog | OFF | | innodb_log_buffer_size | 16777216 | | innodb_log_file_size | 268435456 | | innodb_log_files_in_group | 3 | | innodb_log_group_home_dir | ./ | | innodb_max_dirty_pages_pct | 90 | | innodb_max_purge_lag | 0 | | innodb_mirrored_log_groups | 1 | | innodb_old_blocks_pct | 37 | | innodb_old_blocks_time | 0 | | innodb_open_files | 300 | | innodb_purge_batch_size | 20 | | innodb_purge_threads | 0 | | innodb_read_ahead_threshold | 56 | | innodb_read_io_threads | 8 | | innodb_replication_delay | 0 | | innodb_rollback_on_timeout | OFF | | innodb_spin_wait_delay | 6 | | innodb_stats_on_metadata | ON | | innodb_stats_sample_pages | 8 | | innodb_strict_mode | OFF | | innodb_support_xa | OFF | | innodb_sync_spin_loops | 30 | | innodb_table_locks | ON | | innodb_thread_concurrency | 0 | | innodb_thread_sleep_delay | 10000 | | innodb_use_native_aio | ON | | innodb_use_sys_malloc | ON | | innodb_version | 1.1.2 | | innodb_write_io_threads | 8 | | join_buffer_size | 8388608 | | key_buffer_size | 8388608 | | large_files_support | ON | | large_page_size | 0 | | large_pages | OFF | | last_insert_id | 0 | | max_binlog_cache_size | 18446744073709547520 | | max_binlog_size | 1073741824 | | max_heap_table_size | 536870912 | | max_join_size | 18446744073709551615 | | max_length_for_sort_data | 1024 | | max_sort_length | 1024 | | max_sp_recursion_depth | 0 | | max_tmp_tables | 32 | | optimizer_prune_level | 1 | | optimizer_search_depth | 62 | | optimizer_switch | index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on | | preload_buffer_size | 32768 | | profiling | OFF | | profiling_history_size | 15 | | protocol_version | 10 | | pseudo_thread_id | 13 | | query_alloc_block_size | 8192 | | query_cache_limit | 8388608 | | query_cache_min_res_unit | 4096 | | query_cache_size | 134217728 | | query_cache_type | ON | | query_cache_wlock_invalidate | OFF | | query_prealloc_size | 8192 | | rand_seed1 | 0 | | rand_seed2 | 0 | | range_alloc_block_size | 4096 | | read_buffer_size | 8388608 | | sort_buffer_size | 16777216 | | sql_big_selects | ON | | sql_big_tables | OFF | | sql_max_join_size | 18446744073709551615 | | storage_engine | InnoDB | | tmp_table_size | 536870912 | | tmpdir | /dev/shm | | tx_isolation | REPEATABLE-READ | | version | 5.5.6-rc-log | | version_comment | MySQL Community Server (GPL) | | version_compile_machine | x86_64 | | version_compile_os | linux2.6 | +---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------+
[编辑]感谢Daniel的建议,我已将DATETIME字段更改为DATE。令人高兴的是,该指数现在正在使用。不幸的是,我们仍然在进行文件归档。
+----+-------------+-------+-------+-----------------------------+---------------------+---------+---------------+--------+-----------------------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+-------+-----------------------------+---------------------+---------+---------------+--------+-----------------------------------------------------------+ | 1 | SIMPLE | wpc | index | PRIMARY,WPropertyCube_idx01 | WPropertyCube_idx01 | 8 | NULL | 377500 | Using where; Using index; Using temporary; Using filesort | | 1 | SIMPLE | wmc | ref | PRIMARY,WMeasureCube_idx1 | PRIMARY | 8 | db.wpc.id | 49 | Using where | +----+-------------+-------+-------+-----------------------------+---------------------+---------+---------------+--------+-----------------------------------------------------------+
答案 0 :(得分:0)
我认为你的问题是datetime列actiondate。如果您不需要时间信息,请尝试将此列设置为日期列!那么分组应该更快,你的索引应该可以工作。