Question

我正在处理大量数据输入。我需要知道在新条目插入另一个表后插入/更新表记录的最佳实践。

我认为我们每天的行数超过200万，这是一个持续的过程。

我尝试创建一个触发器，根据复杂的逻辑插入或更新数据。但我怀疑这是一个很好的做法，我担心它会对数据库造成沉重的负担。

我使用PostgreSQL 9.1

当前触发器如下：

DECLARE
  AVar integer;
AdateVar timestamp without time zone;
AnameVar character varying(40);
BEGIN 
 SELECT id  INTO  AVar  FROM table1 WHERE ST_DWithin(NEW.position,ST_SetSRID(ST_MakePoint(longitudedecimal,latitudedecimal ),4326) ,0.01447534783);
 select Adate INTO AdateVar from table2 where id = NEW. id  ORDER BY Adate DESC limit 1;
 IF (aVar > 0) THEN
   select name into AnameVar FROM table1 WHERE id = AVar;
     INSERT into table2 (id,name,date) SELECT  NEW.id,AnameVar,NEW.timestamp;
   ELSE
……….
 END IF;
RETURN NULL;
End;

修改

这里是一个触发器头和功能，功能太长。因为我知道触发器不能调用多个功能！所以它变得非常漫长和复杂，我只发布了12个中的一个案例：

CREATE TRIGGER ts_trigger
  AFTER INSERT
  ON table1
  FOR EACH ROW
  EXECUTE PROCEDURE test_trigger();

declare
Aposition geometry;
Cposition geometry;
Plong double precision;
Plat double precision;
Adate timestamp without time zone;
Cdate timestamp without time zone;
startDate timestamp without time zone;
CnotifDate timestamp without time zone;
AnotifDate timestamp without time zone;
lastmsg timestamp without time zone;
InsideCircle  integer;
InsideSquare  integer;
insidePoint  integer;
distance character varying(40);
-- this variables used to calculate the time in the table3
    inAction character varying(40);
    diff character varying(40);
    days character varying(40);
    hours character varying(40);
    str character varying(40);
    CinAction character varying(40);
    AinAction character varying(40);
BEGIN 


select time_stamp INTO Adate  from table1 where userid=NEW.userid and time_stamp < NEW.time_stamp order by time_stamp desc limit 1 ;
select  position INTO Aposition from table1 where userid=NEW.userid and time_stamp < NEW.time_stamp order by time_stamp desc limit 1;
select time_stamp INTO Cdate  from table1 where userid=NEW.userid and time_stamp > NEW.time_stamp order by time_stamp  limit 1; 
select position INTO Cposition from table1 where userid=NEW.userid and time_stamp > NEW.time_stamp order by time_stamp  limit 1;

SELECT p.num  INTO InsideCircle FROM table3 p WHERE ST_DWithin(Aposition,ST_SetSRID(ST_MakePoint(p.longitudedecimal,p.latitudedecimal ), 4326) ,0.02171302174) ORDER BY ST_Distance(ST_SetSRID(ST_MakePoint(p.longitudedecimal,p.latitudedecimal ), 4326),Aposition) limit 1;
SELECT p.num  INTO InsideSquare FROM table3 p WHERE ST_DWithin(NEW.position ,ST_SetSRID(ST_MakePoint(p.longitudedecimal,p.latitudedecimal ), 4326) ,0.02171302174) ORDER BY ST_Distance(ST_SetSRID(ST_MakePoint(p.longitudedecimal,p.latitudedecimal ), 4326),NEW.position) limit 1;
SELECT p.num  INTO insidePoint  FROM table3 p WHERE ST_DWithin(Cposition,ST_SetSRID(ST_MakePoint(p.longitudedecimal,p.latitudedecimal ), 4326) ,0.02171302174) ORDER BY ST_Distance(ST_SetSRID(ST_MakePoint(p.longitudedecimal,p.latitudedecimal ), 4326),Cposition) limit 1;


-
    IF (InsideCircle >0 and (InsideCircle =InsideSquare or InsideSquare is null))THEN 
    select startDate INTO startDate  from myTable where id=NEW.userid and num = InsideCircle and startDate =Adate; 
        IF (InsideSquare >0)then 
            if (Cdate is not null )then 
                if (insidePoint is null)THEN 
                    diff = NEW.time_stamp  -startDate;
                    str= split_part(diff,' ',2);
                    IF(str = '')then
                    hours= split_part(diff,':',1);
                    days = '0';
                    ELSE
                    str= split_part(diff,' ',3);
                    IF(str = '') then
                    hours ='00';
                    days = split_part(diff,' ',1);
                    ELSE
                    hours= split_part( split_part(diff,' ',3),':',1);
                    days = split_part(diff,' ',1);
                    END IF;
                    END IF;
                    inAction = days || ',' ||hours;
                Update myTable SET notifDate = NEW.time_stamp , time_inAction=inAction WHERE id=NEW.userid  and num =InsideSquare ;
                END IF;

    END IF;
Return Null;
END;

Answer 1

批量更新/插入，使用批次。这样可以减轻数据库的负担，并且如果启用了WAL，则会阻止WAL过载。

什么是有效处理大量数据插入的最佳方法

1 个答案: