Postgresql:在不同的客户端中查询速度慢10倍

时间:2013-06-11 09:34:31

标签: python django postgresql psycopg2

查看postgres服务器日志,我发现从Linux客户端或Windows客户端调用时,同一postgres服务器上的完全相同的查询需要更长的时间(大约长10倍)。

查询来自运行在具有4GB RAM的Linux机器上的Django应用程序以及具有8GB RAM的Windows机器上。两个pyhon环境都有psycopg2库版本2.4.4将请求发送到同一个postgres服务器。

以下是postgres服务器日志

windows查询(带时间):

2013-06-11 12:12:19 EEST [unknown] 10.1.3.152(56895) mferreiraLOG:  duration: 3207.195 ms  statement: SELECT "autotests_tracerperformance"."id", "autotests_tracerperformance"."date", "autotests_tracerperformance"."video_id", "autotests_tracerperformance"."revision_id", "autotests_tracerperformance"."computer_id", "autotests_tracerperformance"."probe", "autotests_tracerperformance"."time_tostart", "autotests_tracerperformance"."hang_atstart", "autotests_tracerperformance"."time_tohang", "autotests_tracerperformance"."hang", "autotests_tracerperformance"."crash", "autotests_tracerperformance"."stacktrace", "autotests_tracerperformance"."framemax", "autotests_tracerperformance"."maxtime", "autotests_tracerperformance"."avgtime" FROM "autotests_tracerperformance" INNER JOIN "revisions" ON ("autotests_tracerperformance"."revision_id" = "revisions"."id") WHERE ("autotests_tracerperformance"."computer_id" = 61  AND "revisions"."repo" = 'Trunk' )

linux查询(更长):

2013-06-11 12:12:56 EEST [unknown] 10.1.3.154(35325) mferreiraLOG:  duration: 22191.773 ms  statement: SELECT "autotests_tracerperformance"."id", "autotests_tracerperformance"."date", "autotests_tracerperformance"."video_id", "autotests_tracerperformance"."revision_id", "autotests_tracerperformance"."computer_id", "autotests_tracerperformance"."probe", "autotests_tracerperformance"."time_tostart", "autotests_tracerperformance"."hang_atstart", "autotests_tracerperformance"."time_tohang", "autotests_tracerperformance"."hang", "autotests_tracerperformance"."crash", "autotests_tracerperformance"."stacktrace", "autotests_tracerperformance"."framemax", "autotests_tracerperformance"."maxtime", "autotests_tracerperformance"."avgtime" FROM "autotests_tracerperformance" INNER JOIN "revisions" ON ("autotests_tracerperformance"."revision_id" = "revisions"."id") WHERE ("autotests_tracerperformance"."computer_id" = 61  AND "revisions"."repo" = 'Trunk' )

直接从psql执行(最快):

2013-06-11 12:19:06 EEST psql [local] mferreiraLOG:  duration: 1332.902 ms  statement: SELECT "autotests_tracerperformance"."id", "autotests_tracerperformance"."date", "autotests_tracerperformance"."video_id", "autotests_tracerperformance"."revision_id", "autotests_tracerperformance"."computer_id", "autotests_tracerperformance"."probe", "autotests_tracerperformance"."time_tostart", "autotests_tracerperformance"."hang_atstart", "autotests_tracerperformance"."time_tohang", "autotests_tracerperformance"."hang", "autotests_tracerperformance"."crash", "autotests_tracerperformance"."stacktrace", "autotests_tracerperformance"."framemax", "autotests_tracerperformance"."maxtime", "autotests_tracerperformance"."avgtime" FROM "autotests_tracerperformance" INNER JOIN "revisions" ON ("autotests_tracerperformance"."revision_id" = "revisions"."id") WHERE ("autotests_tracerperformance"."computer_id" = 61  AND "revisions"."repo" = 'Trunk' );

其他不需要从数据库加载这么多项的查询执行的几乎相同。

为什么此查询的客户端之间存在如此大的时间差异?

注意:传输时间不相关,因为所有计算机都在同一个Intranet中。此外,当客户端请求来自运行postgresql服务器的同一台Linux机器时,可以看到较慢的时间。

Note2 :Psycopg2在Windows和Linux中的安装方式不同。在Windows中,我从预打包的二进制文件安装它,在Linux中我运行'pip install psycopg2',它依赖于系统上可用的postgresql安装。这会导致影响客户端性能的参数的值不同(例如'work_mem'参数)吗?

1 个答案:

答案 0 :(得分:11)

您可能想要检查慢速客户端是否进行SSL加密。默认情况下,它在服务器上设置并且客户端已使用SSL支持进行编译时发生。

对于检索大量数据的查询,时间差异很大。 此外,Debian / Ubuntu等一些Linux发行版默认启用SSL,即使是通过localhost进行TCP连接也是如此。

作为一个例子,这是一个查询检索1,5M行的时间差,这些行的总重量为64Mbytes,带有一个热缓存。

没有加密:

$ psql "host=localhost dbname=mlists sslmode=disable"
Password: 
psql (9.1.7, server 9.1.9)
Type "help" for help.

mlists=> \timing
Timing is on.
mlists=> \o /dev/null
mlists=> select subject from mail;
Time: 1672.258 ms

加密:

$ psql "host=localhost dbname=mlists"
Password: 
psql (9.1.7, server 9.1.9)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.

mlists=> \o /dev/null
mlists=> \timing
Timing is on.
mlists=> select subject from mail;
Time: 7017.935 ms

要全局关闭,可以在SSL=off中设置postgresql.conf

要针对特定​​范围的客户端地址将其关闭,请在更通用的pg_hba.conf条目之前的第一个字段中添加hostnossl host条目。

如果关闭客户端,则取决于驱动程序如何公开sslmode连接参数。如果没有,则如果驱动程序在PGSSLMODE之上实现,则可以使用libpq环境变量。

对于通过Unix域套接字(local)的连接,SSL从不与它们一起使用。