MYSQL:顺序号码表

时间:2013-01-12 21:33:06

标签: mysql sql numbers

我正在尝试获得1到2千万的序列号表。 (或0到2000万)

我很惊讶为这个常见问题找到与MySQL兼容的解决方案有多么困难。

与此类似:Creating a "Numbers Table" in mysql

但答案只有100万。我真的不了解位移计算。

我见过很多SQL的答案,但大多数是针对非MySQL的数据库,所以由于缺乏对MySQL和其他的知识,我不能采用这些代码。

一些参考文献:

SQL, Auxiliary table of numbers

What is the best way to create and populate a numbers table?


请确保您发布的代码在MySQL中兼容并以分号分隔,以便我可以在PhpMyAdmin中运行它。我很感激名为numbers的表名为i

我会对每个解决方案进行基准测试,因此它会被存档并希望在下次有人试图搜索此问题时显示。


迄今为止的基准:

时间是几秒钟。

+---------------+------------------+---------+-----------+------------+
|    Author     |      Method      | 10,000  | 1,000,000 | 20,000,000 |
+---------------+------------------+---------+-----------+------------+
| Devon Bernard | PHP Many Queries | 0.38847 | 39.32716  | ~ 786.54   |
| Bhare         | PHP Few Queries  | 0.00831 | 0.94738   | 19.58823   |
| psadac,Bhare  | LOAD DATA        | 0.00549 | 0.43855   | 10.55236   |
| kjtl          | Bitwise          | 1.36076 | 1.48300   | 4.79226    |
+---------------+------------------+---------+-----------+------------+

8 个答案:

答案 0 :(得分:2)

如果速度是一个问题,你应该根据mysql doc使用比LOAD DATA INFILE更快的INSERT

http://dev.mysql.com/doc/refman/5.5/en/insert-speed.html

When loading a table from a text file, use LOAD DATA INFILE. This is usually 20 times
faster than using INSERT statements. See Section 13.2.6, “LOAD DATA INFILE Syntax”. 

基本上你使用自己喜欢的语言(php?)生成2000万行,然后用LOAD DATA INFILE加载它。

http://dev.mysql.com/doc/refman/5.5/en/load-data.html

答案 1 :(得分:2)

创建这样一个表的典型方法是从:

开始
select 0 as num union all select 1 union all select 2 union all
select 3 union all select 4 union all select 5 union all select 6 union all
select 7 union all select 8 union all select 9

现在,在大多数数据库中,您可以使用with语句并执行以下操作:

with digits as (above query)
select d1.num+10*d2.num+100*d3.num+1000*d4.num+10000*d5.num+100000*d6.num+1000000*d7.num+10000000*87.num as num
from   digits d1 cross join
       digits d2 cross join
       digits d3 cross join
       digits d4 cross join
       digits d5 cross join
       digits d6 cross join
       digits d7 cross join
       (select 0 as num union all select 1) d8

不幸的是,在MySQL中你需要创建一个临时表或重复union all语句:

select d1.num+10*d2.num+100*d3.num+1000*d4.num+10000*d5.num+100000*d6.num+1000000*d7.num+10000000*d7.num as num
from (select 0 as num union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9
     ) d1 cross join
     (select 0 as num union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9
     ) d2 cross join
     (select 0 as num union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9
     ) d3 cross join
     (select 0 as num union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9
     ) d4 cross join
     (select 0 as num union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9
     ) d5 cross join
     (select 0 as num union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9
     ) d6 cross join
     (select 0 as num union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9
     ) d7 cross join
     (select 0 as num union all select 1) d8

在MySQL中,如果要将其放入表中,可以在选择之前使用create table numbers as。但是,不同的数据库具有不同的语法,用于将select的结果转储到表中。

答案 2 :(得分:2)

-- To use the bitwise solution you need a view of 2 to the power 25.
-- the following solution is derived from http://stackoverflow.com/questions/9751318/creating-a-numbers-table-in-mysql
-- the following solution ran in 43.8 seconds with the primary key, without it 4.56 seconds.

-- create a view that has 2 to the power 25 minus 1

-- 2 ^ 1
CREATE or replace VIEW `two_to_the_power_01_minus_1` AS select 0 AS `n` union all select 1 AS `1`;

-- 2 ^ 2
CREATE or replace VIEW `two_to_the_power_02_minus_1` 
AS select
   ((`hi`.`n` << 1) | `lo`.`n`) AS `n`
from (`two_to_the_power_01_minus_1` `lo` join `two_to_the_power_01_minus_1` `hi`) ;

-- 2 ^ 4
CREATE or replace VIEW `two_to_the_power_04_minus_1` 
AS select
   ((`hi`.`n` << 2 ) | `lo`.`n`) AS `n`
from (`two_to_the_power_02_minus_1` `lo` join `two_to_the_power_02_minus_1` `hi`) ;

-- 2 ^ 8
CREATE or replace VIEW `two_to_the_power_08_minus_1` 
AS select
   ((`hi`.`n` << 4 ) | `lo`.`n`) AS `n`
from (`two_to_the_power_04_minus_1` `lo` join `two_to_the_power_04_minus_1` `hi`) ;

-- 2 ^ 12
CREATE or replace VIEW `two_to_the_power_12_minus_1` 
AS select
   ((`hi`.`n` << 8 ) | `lo`.`n`) AS `n`
from (`two_to_the_power_08_minus_1` `lo` join `two_to_the_power_04_minus_1` `hi`) ;

-- 2 ^ 13
CREATE or replace VIEW `two_to_the_power_13_minus_1`
AS select
   ((`hi`.`n` << 1) | `lo`.`n`) AS `n`
from (`two_to_the_power_01_minus_1` `lo` join `two_to_the_power_12_minus_1` `hi`);



-- create a table to store the interim results for speed of retrieval
drop table if exists numbers_2_to_the_power_13_minus_1;

create table `numbers_2_to_the_power_13_minus_1` (
  `i` int(11) unsigned
) ENGINE=myisam DEFAULT CHARSET=latin1 ;

-- faster 2 ^ 13
insert into numbers_2_to_the_power_13_minus_1( i )
select n from `two_to_the_power_13_minus_1` ;

-- faster 2 ^ 12
CREATE or replace view `numbers_2_to_the_power_12_minus_1`
AS select
   `numbers_2_to_the_power_13_minus_1`.`i` AS `i`
from `numbers_2_to_the_power_13_minus_1`
where (`numbers_2_to_the_power_13_minus_1`.`i` < (1 << 12));

-- faster 2 ^ 25
CREATE or replace VIEW `numbers_2_to_the_power_25_minus_1`
AS select
   ((`hi`.`i` << 12) | `lo`.`i`) AS `i`
from (`numbers_2_to_the_power_12_minus_1` `lo` join `numbers_2_to_the_power_13_minus_1` `hi`);

-- create table for results

drop table if exists numbers ;

create table `numbers` (
  `i` int(11) signed 
  , primary key(`i`)
) ENGINE=myisam DEFAULT CHARSET=latin1;

-- insert the numbers
insert into numbers(i)
select i from numbers_2_to_the_power_25_minus_1
where i <= 20000000 ;

drop view if exists numbers_2_to_the_power_25_minus_1 ;
drop view if exists numbers_2_to_the_power_12_minus_1 ;
drop table if exists numbers_2_to_the_power_13_minus_1 ;
drop view if exists two_to_the_power_13_minus_1 ;
drop view if exists two_to_the_power_12_minus_1 ;
drop view if exists two_to_the_power_08_minus_1 ;
drop view if exists two_to_the_power_04_minus_1 ;
drop view if exists two_to_the_power_02_minus_1 ;
drop view if exists two_to_the_power_01_minus_1 ;

答案 3 :(得分:0)

我不确定你是否打算拨打一个电话拨打2000万行,或者拨打电话拨打2000万次电话。第二种情况的一个例子是:

<?php
$i =0;
while($i <= 20000000){
$sql = mysql_query("INSERT INTO table_name VALUES ('$i')");
$i +=1;
}
?>

如果您正在寻找SQL解决方案,则无法尝试适应

DROP TABLE NumbersTest
DECLARE @RunDate datetime
SET @RunDate=GETDATE()
SELECT TOP 20000000 IDENTITY(int,1,1) AS Number
    INTO NumbersTest
    FROM sys.objects s1
    CROSS JOIN sys.objects s2
ALTER TABLE NumbersTest ADD CONSTRAINT PK_NumbersTest PRIMARY KEY CLUSTERED (Number)
PRINT CONVERT(varchar(20),datediff(ms,@RunDate,GETDATE()))+' milliseconds'
SELECT COUNT(*) FROM NumbersTest

从此post获取并报告平均产生10,000行,平均为56.3毫秒。

答案 4 :(得分:0)

为了回应Devon Bernard的回答,我决定使用PDO Mysql PHP来处理它并使用几个查询的概念。起初我尝试用一​​个大的查询来做这件事,但是PHP用默认设置耗尽了内存,所以我决定调整每10万次运行一次。即使你分配了足够的内存来保持,也没有明显的改进。

$i = 1;
$inserts = array();
while($i <= 20000000) {
    $inserts[] = "($i)";

    if($i % 100000 == 0) {
        $dbh->beginTransaction();
        $query = "INSERT INTO numbers(i) VALUES " . implode(',', $inserts) . ";";
            $sth = $dbh->prepare($query);
            $sth->execute();
        $dbh->commit();
        $inserts = array();
    }
    $i +=1;
}

答案 5 :(得分:0)

如果已经引用了这个答案,请道歉。我的机器花了18.79秒(戴尔笔记本电脑,如果重要的话)......

它改编自http://datacharmer.blogspot.co.uk/2006/06/filling-test-tables-quickly.html的旧解决方案,但至关重要的是,这不适用于默认的'vanilla'InnoDB引擎,如果你尝试在一开始就建立PK,它会慢得多。

在正面,你可以免费获得额外的13.5 M行!

drop table if exists numbers;
create table numbers ( id int not null) engine = myisam;

delimiter $$

drop procedure if exists fill_numbers $$
create procedure fill_numbers()
deterministic
begin
  declare counter int default 1;
  insert into numbers values (1);
  while counter < 20000000
  do
      insert into numbers (id)
          select id + counter
          from numbers;
      select count(*) into counter from numbers;
      select counter;
  end while;
end $$
delimiter ;

call fill_numbers();

答案 6 :(得分:0)

习惯psadac使用LOAD DATA INFILE的答案以及应用于fwrite的BULK插入的想法:

$fh = fopen("data_num.txt", 'a') or die("can't open file");
$i =1;
while($i <= 20000000) {
    $num_string .= "$i\n";
    if($i % 1000000 == 0) {
        fwrite($fh, $num_string);
        $num_string = "";
    }
    $i +=1;
}
fclose($fh);

$dbh->beginTransaction();
$query = "LOAD DATA INFILE '" . addslashes(realpath("data_num.txt")) . "' INTO TABLE numbers LINES TERMINATED BY '\n';";
    $sth = $dbh->prepare($query);
    $sth->execute();
$dbh->commit();
unlink("data_num.txt");

我必须使用addslashes,因为我使用的是windows环境。

值得注意的是,通过写入20次以上文件超过2000万次来执行BULK技术导致约10秒,相比之下只需写入2000万次就可以达到约75秒。使用字符串连接而不是将值推入数组并且内爆几乎快两倍。

答案 7 :(得分:0)

更简单&amp;更快的解决方案

(原始代码here

CREATE TABLE `numbers` (
  `i` INT(11) SIGNED 
  , PRIMARY KEY(`i`)
) ENGINE=myisam DEFAULT CHARSET=latin1;

INSERT INTO numbers(i) SELECT @row := @row + 1 FROM 
(SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t,
(SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t2, 
(SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t3, 
(SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t4, 
(SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t5, 
(SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t6, 
(SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t7, 
(SELECT 0 UNION ALL SELECT 1) t8, 
(SELECT @row:=0) ti;

在安装了MySQL 5.5.29的笔记本电脑上,将the accepted answer与这个比较:

+-----------------+-------+---------------+
| Method          | Rows  | Time consumed |
+-----------------+-------+---------------+
| Accepted answer | 20M+1 |         42.4s |
+-----------------+-------+---------------+
| This one        | 20M   |         35.9s |
+-----------------+-------+---------------+

减少约15%的时间,没有中间视图或表格,更易于阅读。