在一列中搜索950个数字的最佳方式是什么?

时间:2017-01-19 12:16:43

标签: sqlite full-text-search

我在sqlite3数据库中有数百万行。专栏分数'单行包含以下示例:

{1399808086,1366221142,1374614902,1374608759,1375598069,1375270116,1935207612,1914502332,1913478333,1930188205,1934563311,1942881023,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988 ... up to nearly 1'000 numbers, ending with }

搜索列'指向'的行的最佳方法是什么?包括
a)所有三个示例搜索值 1366221142,1374614902,1374608759
(上述内容中的位置二,三和四)

b)尽可能多(3或2或1)上述3个示例搜索值

我使用索引进行了尝试,但搜索了LIKE和'%1366221142%'需要"永远"。

实际上我是用FTS5尝试的,但导入新创建的虚拟表似乎需要几天时间。

你知道其他任何可能吗?

1 个答案:

答案 0 :(得分:1)

如果你可以使用Python,那么这种方法非常快#'

import sqlite3
conn = sqlite3.connect(':memory:')

points = '{1399808086,1366221142,1374614902,1374608759,1375598069,1375270116,1935207612,1914502332,1913478333,1930188205,1934563311,1942881023,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860}'

conn.execute('CREATE TABLE something (recno, points)')

for r in range(10000):
    conn.execute('INSERT INTO something (recno, points) values (?,?)', (r, points))

seeking = ['1366221142', '1374614902', '1374608759']
first = True
for row in conn.execute('SELECT recno, points FROM something'):
    pointsList = points[1:-1].split(',')
    counts = { _:pointsList.count(_) for _ in seeking }
    if first:
        print (row)
        print (counts)  
        first = False

(删节)输出是:

(0,' {1399808086,1366221142,1374614902,1374608759,1375598069,1375270116,1935207612,1914502332,1913478333,1930188205,1934563311,1942881023,1373508175,-778100129,-788765075,-788763091,-790856156, - 790835404,-791756027,-795938489,-779165370,... 324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260, 1913458860}') {' 1374614902':1,' 1374608759':1,' 1366221142':1}

请注意,代码会安排将10,000个字符串副本(扩展到1,000个数字)放入数据库,然后处理它们。当然,数据库在内存中,这是一个需要考虑的因素。

你可以尝试一下。