Question

我有一个简单的表 id_exp ，其中有1.3亿条记录，PostgreSQL 11。

ID格式-'0123456789'，10位数字。

postgresql.conf （总RAM 4Gb）
  shared_buffers = 512MB
  temp_buffers = 128MB
  work_mem = 16MB

 CREATE TABLE id_exp (
    sn bigint NOT NULL,
    dt date NOT NULL,
    lupdate date NOT NULL,
    CONSTRAINT passid_exp_pkey PRIMARY KEY (sn)
) PARTITION BY RANGE (sn) WITH ( OIDS = FALSE ) TABLESPACE pg_default;

CREATE TABLE public.id_exp_00 PARTITION OF public.id_exp FOR VALUES FROM ('0') TO ('499999999');
CREATE TABLE public.id_exp_05 PARTITION OF public.id_exp FOR VALUES FROM ('500000000') TO ('999999999');
.....
CREATE TABLE public.id_exp_95 PARTITION OF public.id_exp FOR VALUES FROM ('9500000000') TO ('9999999999');

select count(*) from only id_exp;

count = 0

select count(*) from id_exp;

count = 128725653

选择查询非常快：

select * from passid_exp where sn=3;

查询总运行时间为125毫秒。

但是UPSERT却步履蹒跚。

我的代码（psql -f upd.sql）：

START TRANSACTION;
CREATE TEMP TABLE tmp_tbl (sn bigint) ON COMMIT DROP;

\COPY tmp_tbl FROM tmp_fms.csv With CSV;

ALTER TABLE tmp_tbl ADD CONSTRAINT tmp_tbl_idx PRIMARY KEY (sn);

DO $$
DECLARE
 tbl integer;
BEGIN

FOR tbl IN SELECT i FROM generate_series(00, 95, 5) AS t(i)
LOOP
  EXECUTE format('INSERT INTO passid_exp_%s (sn,dt,lupdate) (SELECT sn, current_date dt, current_date lupdate from tmp_tbl where sn >= %s00000000 and sn <= %s99999999) ON conflict (sn) DO update set lupdate=current_date', lpad(tbl::text,2,'0'),lpad(tbl::text,2,'0'),lpad((tbl+4)::text,2,'0'));
END LOOP;
END; $$;

COMMIT;

工作了30分钟！！！只有50万条记录。

16.12 20:01:08 Start processing
START TRANSACTION
CREATE TABLE
COPY 500000
ALTER TABLE
DO
INSERT 0 0
COMMIT
16.12 20:33:29 End processing

有什么想法吗？

PostgreSQL-Upsert大表非常慢

0 个答案: