PostgreSQL计算加载插入时查询和触发器的总执行时间

时间:2016-07-03 18:47:25

标签: python postgresql psycopg2

我想测量PostgresSQL执行一系列触发器所消耗的总时间。

简而言之,系统是一个python程序,它使用DB来存储和计算使用postgresql中安装的一些插件的最短路径。系统依赖于在请求表中插入请求后调用的触发器。

   for i in range(50):
        readyhostslist = random.sample(allhosts, 100)
        fid = 0
        fw = 0
        hostlistready = {}
        for n in xrange(0,len(readyhostslist), 2):
            hostlistready[readyhostslist[n][0]] = readyhostslist[n+1][0]
        for src, dst in hostlistready.iteritems():
            try:
                # get next flow id

                #print src, dst
                self.db.cursor.execute("SELECT * FROM rm;")
                fid = len(self.db.cursor.fetchall()) + 1
                self.db.cursor.execute("INSERT INTO tracktime (fid) VALUES ({0} )".format(fid))
                self.db.cursor.execute("INSERT INTO rm (fid, src, dst) VALUES ({0}, {1}, {2});".format(fid, src, dst))
                self.db.cursor.execute("UPDATE rm set FW = {0} where fid = {1};".format(fw, fid))
                self.db._conn.commit()
            except Exception, e:
                print "Failure: flow not installed --", e

我想要做的是知道从使用python postgresql驱动程序插入请求到结果存储在其他表中的那一刻所经过的总时间。

我尝试了以下但是我失败了。我在插入请求之前开始插入当前时间,并修改最后一个被调用的触发器以更新结束时间。

DROP TABLE IF EXISTS tracktime CASCADE;
CREATE UNLOGGED TABLE tracktime (
       id      serial ,
       fid      integer,
       timestart timestamp default clock_timestamp(),
       timeend  timestamp ,
       waqtstart timestamp ,
       waqtend  timestamp ,
       PRIMARY KEY (id)
);
DROP TABLE IF EXISTS p_routing CASCADE;
CREATE UNLOGGED TABLE p_routing (
    counts    integer,
    status    text,
    PRIMARY key (counts)
);

CREATE TRIGGER run_routing_trigger
     AFTER INSERT ON p_routing
     FOR EACH ROW
   EXECUTE PROCEDURE spv_constraint1_fun();

CREATE OR REPLACE RULE run_routing AS
    ON INSERT TO p_routing
    WHERE (NEW.status = 'on')
    DO ALSO (
         UPDATE p_routing SET status = 'off' WHERE counts = NEW.counts;
         );



DROP TABLE IF EXISTS p_{0} CASCADE;
CREATE UNLOGGED TABLE p_{0} (
       counts integer,
       status text,
       PRIMARY key (counts)
);


CREATE OR REPLACE RULE run_{0} AS
    ON INSERT TO p_{0}
    WHERE (NEW.status = 'on')
    DO ALSO (
        DELETE FROM {1};
        UPDATE p_{0} SET status = 'off' WHERE counts = NEW.counts;
        );

CREATE OR REPLACE FUNCTION spv_constraint1_fun ()
RETURNS TRIGGER
AS $$
plpy.notice ("spv_constraint1_fun")
if TD["new"]["status"] == 'on':
    rm = plpy.execute ("SELECT * FROM rm_delta;")
    flowid = 0
    for t in rm:
        if t["isadd"] == 1:
            f = t["fid"]
            flowid = f
            s = t["src"]
            d = t["dst"]
            pv = plpy.execute("""SELECT array(SELECT id1 FROM pgr_dijkstra('SELECT 1 as id, sid as source, nid as target, 1.0::float8 as cost FROM tp WHERE isactive = 1',""" +str (s) + "," + str (d)  + ",FALSE, FALSE))""")[0]['array']
            plpy.execute ("UPDATE tracktime  SET timeend = clock_timestamp() where fid =" + str (flowid) +";")
            l = len (pv)
            plpy.execute ("INSERT INTO trackzeit (fid) VALUES(" + str (flowid) +");")
            for i in range (l):
                if i + 2 < l:
                    plpy.execute ("INSERT INTO cf (fid,pid,sid,nid) VALUES (" + str (f) + "," + str (pv[i]) + "," +str (pv[i+1]) +"," + str (pv[i+2])+  ");")
            plpy.execute ("UPDATE trackzeit  SET zeitend = clock_timestamp() where fid =" + str (flowid) +";")
        elif t["isadd"] == 0:
            f = t["fid"]
            plpy.execute ("DELETE FROM cf WHERE fid =" +str (f) +";")

    plpy.execute ("DELETE FROM rm_delta;")

return None;
$$ LANGUAGE 'plpythonu' VOLATILE SECURITY DEFINER;


CREATE TRIGGER spv_constraint1
       AFTER INSERT ON p_spv
       FOR EACH ROW
       EXECUTE PROCEDURE spv_constraint1_fun();

虽然这适用于单个请求,但当我尝试循环并插入100个请求时,它不会。我观察到的是python驱动程序或数据库插入100个请求然后触发每个请求的函数。

P.S。我在python程序中的每个插入语句后尝试自动提交,但仍然遇到同样的问题。

这是结果的引用,您可以看到累积或接近累积(其他一些试验将没有这种累积现象)。每行代表使用与流程数(fid)相关联的PostgreSQL函数测量的总时间

77.625
130.395
168.6
208.572
239.531
273.252
295.924
320.999
349.042
369.552
398.883
432.04
462.708
498.156
525.315
565.321
601.193
636.402
667.983
695.893
731.778
759.155
792.391
814.026
842.942
874.946
917.369
934.897
963.269

0 个答案:

没有答案