我有一个MySQL表（rangeTest），它有超过1000万行数据，我需要在Web应用程序中不断查询，因此需要非常快。每行都包含一个整数范围，如列rangeStartInt和rangeEndInt所描述。没有整数范围在行与行之间重叠，并且在行之间同时运行。因此，例如，一系列5行将如下所示（它们是下面数据中的前两列）：
3758080000,3758084095,1835841,1835841,,0,0,,37.5112,126.9741,200
3758084096,3758085119,1835841,1835841,,0,0,,37.5112,126.9741,200
3758085120,3758086143,1841811,1835841,,0,0,,35.1547,126.9156,1
3758086144,3758088191,1835841,1835841,,0,0,,37.5112,126.9741,200
3758088192,3758088447,8314434,1643084,,0,0,,1.2000,104.0000,100
每个查询都根据整数返回一行。例如，整数值3758085130将产生以下查询：
SELECT * FROM rangeTest WHERE rangeStartInt <= 3758085130 AND rangeEndInt >= 3758085130
这将返回以下单行：
3758085120,3758086143,1841811,1835841,,0,0,,35.1547,126.9156,1
在未分区的形式中，rangeTest表上的每个查询大约需要8秒。因此我添加了RANGE分区如下：
CREATE TABLE `rangeTest` (
`rangeStartInt` bigint(11) NOT NULL,
`rangeEndInt` bigint(11) NOT NULL,
`a` bigint(11) NOT NULL,
`b` varchar(11) COLLATE utf8_unicode_ci DEFAULT NULL,
`c` varchar(11) COLLATE utf8_unicode_ci DEFAULT NULL,
`d` varchar(1) COLLATE utf8_unicode_ci DEFAULT NULL,
`e` varchar(1) COLLATE utf8_unicode_ci DEFAULT NULL,
`f` varchar(10) COLLATE utf8_unicode_ci DEFAULT NULL,
`g` varchar(10) COLLATE utf8_unicode_ci DEFAULT NULL,
`h` varchar(10) COLLATE utf8_unicode_ci DEFAULT NULL,
`i` varchar(5) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`rangeStartInt`,`rangeEndInt`,`a`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
PARTITION BY RANGE COLUMNS(rangeStartInt,rangeEndInt)
(
PARTITION p0 VALUES LESS THAN (8000000,8000000) ENGINE = InnoDB,
PARTITION p1 VALUES LESS THAN (16000000,16000000) ENGINE = InnoDB,
PARTITION p2 VALUES LESS THAN (24000000,24000000) ENGINE = InnoDB,
PARTITION p3 VALUES LESS THAN (32000000,32000000) ENGINE = InnoDB,
PARTITION p4 VALUES LESS THAN (40000000,40000000) ENGINE = InnoDB,
PARTITION p5 VALUES LESS THAN (48000000,48000000) ENGINE = InnoDB,
PARTITION p6 VALUES LESS THAN (56000000,56000000) ENGINE = InnoDB,
PARTITION p7 VALUES LESS THAN (64000000,64000000) ENGINE = InnoDB,
PARTITION p8 VALUES LESS THAN (72000000,72000000) ENGINE = InnoDB,
PARTITION p9 VALUES LESS THAN (80000000,80000000) ENGINE = InnoDB,
PARTITION p10 VALUES LESS THAN (88000000,88000000) ENGINE = InnoDB,
PARTITION p11 VALUES LESS THAN (96000000,96000000) ENGINE = InnoDB,
PARTITION p12 VALUES LESS THAN (104000000,104000000) ENGINE = InnoDB,
PARTITION p13 VALUES LESS THAN (112000000,112000000) ENGINE = InnoDB,
PARTITION p14 VALUES LESS THAN (120000000,120000000) ENGINE = InnoDB,
PARTITION p15 VALUES LESS THAN (128000000,128000000) ENGINE = InnoDB,
PARTITION p16 VALUES LESS THAN (136000000,136000000) ENGINE = InnoDB,
PARTITION p17 VALUES LESS THAN (144000000,144000000) ENGINE = InnoDB,
PARTITION p18 VALUES LESS THAN (152000000,152000000) ENGINE = InnoDB,
PARTITION p19 VALUES LESS THAN (160000000,160000000) ENGINE = InnoDB,
PARTITION p20 VALUES LESS THAN (168000000,168000000) ENGINE = InnoDB,
PARTITION p21 VALUES LESS THAN (176000000,176000000) ENGINE = InnoDB,
PARTITION p22 VALUES LESS THAN (184000000,184000000) ENGINE = InnoDB,
PARTITION p23 VALUES LESS THAN (192000000,192000000) ENGINE = InnoDB,
PARTITION p24 VALUES LESS THAN (200000000,200000000) ENGINE = InnoDB,
PARTITION p25 VALUES LESS THAN (208000000,208000000) ENGINE = InnoDB,
PARTITION p26 VALUES LESS THAN (216000000,216000000) ENGINE = InnoDB,
PARTITION p27 VALUES LESS THAN (224000000,224000000) ENGINE = InnoDB,
PARTITION p28 VALUES LESS THAN (232000000,232000000) ENGINE = InnoDB,
PARTITION p29 VALUES LESS THAN (240000000,240000000) ENGINE = InnoDB,
PARTITION p30 VALUES LESS THAN (248000000,248000000) ENGINE = InnoDB,
PARTITION p31 VALUES LESS THAN (256000000,256000000) ENGINE = InnoDB,
PARTITION p32 VALUES LESS THAN (264000000,264000000) ENGINE = InnoDB,
PARTITION p33 VALUES LESS THAN (272000000,272000000) ENGINE = InnoDB,
PARTITION p34 VALUES LESS THAN (280000000,280000000) ENGINE = InnoDB,
PARTITION p35 VALUES LESS THAN (288000000,288000000) ENGINE = InnoDB,
PARTITION p36 VALUES LESS THAN (296000000,296000000) ENGINE = InnoDB,
PARTITION p37 VALUES LESS THAN (304000000,304000000) ENGINE = InnoDB,
PARTITION p38 VALUES LESS THAN (312000000,312000000) ENGINE = InnoDB,
PARTITION p39 VALUES LESS THAN (320000000,320000000) ENGINE = InnoDB,
PARTITION p40 VALUES LESS THAN (328000000,328000000) ENGINE = InnoDB,
PARTITION p41 VALUES LESS THAN (336000000,336000000) ENGINE = InnoDB,
PARTITION p42 VALUES LESS THAN (344000000,344000000) ENGINE = InnoDB,
PARTITION p43 VALUES LESS THAN (352000000,352000000) ENGINE = InnoDB,
PARTITION p44 VALUES LESS THAN (360000000,360000000) ENGINE = InnoDB,
PARTITION p45 VALUES LESS THAN (368000000,368000000) ENGINE = InnoDB,
PARTITION p46 VALUES LESS THAN (376000000,376000000) ENGINE = InnoDB,
PARTITION p47 VALUES LESS THAN (384000000,384000000) ENGINE = InnoDB,
PARTITION p48 VALUES LESS THAN (392000000,392000000) ENGINE = InnoDB,
PARTITION p49 VALUES LESS THAN (400000000,400000000) ENGINE = InnoDB,
PARTITION p50 VALUES LESS THAN (408000000,408000000) ENGINE = InnoDB,
PARTITION p51 VALUES LESS THAN (416000000,416000000) ENGINE = InnoDB,
PARTITION p52 VALUES LESS THAN (424000000,424000000) ENGINE = InnoDB,
PARTITION p53 VALUES LESS THAN (432000000,432000000) ENGINE = InnoDB,
PARTITION p54 VALUES LESS THAN (440000000,440000000) ENGINE = InnoDB,
PARTITION p55 VALUES LESS THAN (448000000,448000000) ENGINE = InnoDB,
PARTITION p56 VALUES LESS THAN (456000000,456000000) ENGINE = InnoDB,
PARTITION p57 VALUES LESS THAN (464000000,464000000) ENGINE = InnoDB,
PARTITION p58 VALUES LESS THAN (472000000,472000000) ENGINE = InnoDB,
PARTITION p59 VALUES LESS THAN (480000000,480000000) ENGINE = InnoDB,
PARTITION p60 VALUES LESS THAN (488000000,488000000) ENGINE = InnoDB,
PARTITION p61 VALUES LESS THAN (496000000,496000000) ENGINE = InnoDB,
PARTITION p62 VALUES LESS THAN (504000000,504000000) ENGINE = InnoDB,
PARTITION p63 VALUES LESS THAN (512000000,512000000) ENGINE = InnoDB,
PARTITION p64 VALUES LESS THAN (520000000,520000000) ENGINE = InnoDB,
PARTITION p65 VALUES LESS THAN (528000000,528000000) ENGINE = InnoDB,
PARTITION p66 VALUES LESS THAN (536000000,536000000) ENGINE = InnoDB,
PARTITION p67 VALUES LESS THAN (544000000,544000000) ENGINE = InnoDB,
PARTITION p68 VALUES LESS THAN (552000000,552000000) ENGINE = InnoDB,
PARTITION p69 VALUES LESS THAN (560000000,560000000) ENGINE = InnoDB,
PARTITION p70 VALUES LESS THAN (568000000,568000000) ENGINE = InnoDB,
PARTITION p71 VALUES LESS THAN (576000000,576000000) ENGINE = InnoDB,
PARTITION p72 VALUES LESS THAN (584000000,584000000) ENGINE = InnoDB,
PARTITION p73 VALUES LESS THAN (592000000,592000000) ENGINE = InnoDB,
PARTITION p74 VALUES LESS THAN (600000000,600000000) ENGINE = InnoDB,
PARTITION p75 VALUES LESS THAN (608000000,608000000) ENGINE = InnoDB,
PARTITION p76 VALUES LESS THAN (616000000,616000000) ENGINE = InnoDB,
PARTITION p77 VALUES LESS THAN (624000000,624000000) ENGINE = InnoDB,
PARTITION p78 VALUES LESS THAN (632000000,632000000) ENGINE = InnoDB,
PARTITION p79 VALUES LESS THAN (640000000,640000000) ENGINE = InnoDB,
PARTITION p80 VALUES LESS THAN (648000000,648000000) ENGINE = InnoDB,
PARTITION p81 VALUES LESS THAN (656000000,656000000) ENGINE = InnoDB,
PARTITION p82 VALUES LESS THAN (664000000,664000000) ENGINE = InnoDB,
PARTITION p83 VALUES LESS THAN (672000000,672000000) ENGINE = InnoDB,
PARTITION p84 VALUES LESS THAN (680000000,680000000) ENGINE = InnoDB,
PARTITION p85 VALUES LESS THAN (688000000,688000000) ENGINE = InnoDB,
PARTITION p86 VALUES LESS THAN (696000000,696000000) ENGINE = InnoDB,
PARTITION p87 VALUES LESS THAN (704000000,704000000) ENGINE = InnoDB,
PARTITION p88 VALUES LESS THAN (712000000,712000000) ENGINE = InnoDB,
PARTITION p89 VALUES LESS THAN (720000000,720000000) ENGINE = InnoDB,
PARTITION p90 VALUES LESS THAN (728000000,728000000) ENGINE = InnoDB,
PARTITION p91 VALUES LESS THAN (736000000,736000000) ENGINE = InnoDB,
PARTITION p92 VALUES LESS THAN (744000000,744000000) ENGINE = InnoDB,
PARTITION p93 VALUES LESS THAN (752000000,752000000) ENGINE = InnoDB,
PARTITION p94 VALUES LESS THAN (760000000,760000000) ENGINE = InnoDB,
PARTITION p95 VALUES LESS THAN (768000000,768000000) ENGINE = InnoDB,
PARTITION p96 VALUES LESS THAN (776000000,776000000) ENGINE = InnoDB,
PARTITION p97 VALUES LESS THAN (784000000,784000000) ENGINE = InnoDB,
PARTITION p98 VALUES LESS THAN (792000000,792000000) ENGINE = InnoDB,
PARTITION p99 VALUES LESS THAN (800000000,800000000) ENGINE = InnoDB,
PARTITION p100 VALUES LESS THAN (808000000,808000000) ENGINE = InnoDB,
PARTITION p101 VALUES LESS THAN (816000000,816000000) ENGINE = InnoDB,
PARTITION p102 VALUES LESS THAN (824000000,824000000) ENGINE = InnoDB,
PARTITION p103 VALUES LESS THAN (832000000,832000000) ENGINE = InnoDB,
PARTITION p104 VALUES LESS THAN (840000000,840000000) ENGINE = InnoDB,
PARTITION p105 VALUES LESS THAN (848000000,848000000) ENGINE = InnoDB,
PARTITION p106 VALUES LESS THAN (856000000,856000000) ENGINE = InnoDB,
PARTITION p107 VALUES LESS THAN (864000000,864000000) ENGINE = InnoDB,
PARTITION p108 VALUES LESS THAN (872000000,872000000) ENGINE = InnoDB,
PARTITION p109 VALUES LESS THAN (880000000,880000000) ENGINE = InnoDB,
...
PARTITION p497 VALUES LESS THAN (3984000000,3984000000) ENGINE = InnoDB,
PARTITION p498 VALUES LESS THAN (3992000000,3992000000) ENGINE = InnoDB
)
在表格中生成500个分区。这适用于INT值的任何显式SELECT搜索，如下所示：
explain partitions select * from rangeTest where rangeStartInt = 3758084096 AND rangeEndInt = 3758085119;
+----+-------------+------------------------------+------------+------+---------------+---------+---------+-------------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+------------------------------+------------+------+---------------+---------+---------+-------------+------+----------+-------+
| 1 | SIMPLE | rangeTest | p469 | ref | PRIMARY | PRIMARY | 16 | const,const | 1 | 100.00 | NULL |
+----+-------------+------------------------------+------------+------+---------------+---------+---------+-------------+------+----------+-------+
使用相关分区并且相关行返回大约0.007秒。然而，当我将少于和多于运算符添加到混合中时
explain partitions select * from rangeTest where rangeStartInt <= 3758084096 AND rangeEndInt >= 3758085119;
+----+-------------+------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+---------------+---------+---------+------+---------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+---------------+---------+---------+------+---------+----------+-------------+
| 1 | SIMPLE | rangeTest | p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20,p21,p22,p23,p24,p25,p26,p27,p28,p29,p30,p31,p32,p33,p34,p35,p36,p37,p38,p39,p40,p41,p42,p43,p44,p45,p46,p47,p48,p49,p50,p51,p52,p53,p54,p55,p56,p57,p58,p59,p60,p61,p62,p63,p64,p65,p66,p67,p68,p69,p70,p71,p72,p73,p74,p75,p76,p77,p78,p79,p80,p81,p82,p83,p84,p85,p86,p87,p88,p89,p90,p91,p92,p93,p94,p95,p96,p97,p98,p99,p100,p101,p102,p103,p104,p105,p106,p107,p108,p109,p110,p111,p112,p113,p114,p115,p116,p117,p118,p119,p120,p121,p122,p123,p124,p125,p126,p127,p128,p129,p130,p131,p132,p133,p134,p135,p136,p137,p138,p139,p140,p141,p142,p143,p144,p145,p146,p147,p148,p149,p150,p151,p152,p153,p154,p155,p156,p157,p158,p159,p160,p161,p162,p163,p164,p165,p166,p167,p168,p169,p170,p171,p172,p173,p174,p175,p176,p177,p178,p179,p180,p181,p182,p183,p184,p185,p186,p187,p188,p189,p190,p191,p192,p193,p194,p195,p196,p197,p198,p199,p200,p201,p202,p203,p204,p205,p206,p207,p208,p209,p210,p211,p212,p213,p214,p215,p216,p217,p218,p219,p220,p221,p222,p223,p224,p225,p226,p227,p228,p229,p230,p231,p232,p233,p234,p235,p236,p237,p238,p239,p240,p241,p242,p243,p244,p245,p246,p247,p248,p249,p250,p251,p252,p253,p254,p255,p256,p257,p258,p259,p260,p261,p262,p263,p264,p265,p266,p267,p268,p269,p270,p271,p272,p273,p274,p275,p276,p277,p278,p279,p280,p281,p282,p283,p284,p285,p286,p287,p288,p289,p290,p291,p292,p293,p294,p295,p296,p297,p298,p299,p300,p301,p302,p303,p304,p305,p306,p307,p308,p309,p310,p311,p312,p313,p314,p315,p316,p317,p318,p319,p320,p321,p322,p323,p324,p325,p326,p327,p328,p329,p330,p331,p332,p333,p334,p335,p336,p337,p338,p339,p340,p341,p342,p343,p344,p345,p346,p347,p348,p349,p350,p351,p352,p353,p354,p355,p356,p357,p358,p359,p360,p361,p362,p363,p364,p365,p366,p367,p368,p369,p370,p371,p372,p373,p374,p375,p376,p377,p378,p379,p380,p381,p382,p383,p384,p385,p386,p387,p388,p389,p390,p391,p392,p393,p394,p395,p396,p397,p398,p399,p400,p401,p402,p403,p404,p405,p406,p407,p408,p409,p410,p411,p412,p413,p414,p415,p416,p417,p418,p419,p420,p421,p422,p423,p424,p425,p426,p427,p428,p429,p430,p431,p432,p433,p434,p435,p436,p437,p438,p439,p440,p441,p442,p443,p444,p445,p446,p447,p448,p449,p450,p451,p452,p453,p454,p455,p456,p457,p458,p459,p460,p461,p462,p463,p464,p465,p466,p467,p468,p469 | range | PRIMARY | PRIMARY | 8 | NULL | 5048679 | 33.33 | Using where |
+----+-------------+------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+---------------+---------+---------+------+---------+----------+-------------+
我回到了8s查询。所以我的问题是如何避免这种情况？我想我需要在我的应用程序（Python）中使用一个数组来计算我需要查询的分区，然后才能通过请求该特定分区来访问数据库。这可以工作并将查询时间调回到大约0.008s，但由于表格将通过CSV导入定期更新，因此它看起来非常笨重，因此顶端范围可能会发生变化，需要更改Python数组以适应这种情况并避免错误
我是否正在将分区过滤从MySQL中移除并在Python代码中处理此问题？或者MySQL中有什么东西我在分区表的大于和小于查询时缺少？
或者是否可以在查询中从MySQl中检索分区范围，然后我可以在Python代码中使用它来计算要在查询中使用的相关分区？
感谢。
答案 0 :(得分：0)
您有非重叠的整数范围作为主＆＃34;键＆＃34;进入表格。 MySQL没有这种空间的情况。事实上，现有的工具都没有做得好。甚至不是PARTITION。
在8.0版之前，在表中有500个分区是不切实际的。 50是实际限制;之后会出现各种低效率。
表上没有简单的索引（分区与否）会比（通常）扫描半表更好。
但是有一个解决方案。有关如何有效处理IP地址，请参阅我的blog。您需要根据您的数字类型进行调整（并删除INET_ATON类型的代码）。
使用我的代码，典型的查询只会触及所需的行;不是桌子的一半。