Question

我们有一个应用程序通过psqlodbc 09.00.0200驱动程序以下列方式从PostgreSQL 9.0.3数据库获取一些数据：

1）带有START TRANSACTION的SQLExecDirect 2）SQLExecDirect with

DECLARE foo SCROLL CURSOR FOR  
SELECT table.alotofcolumns  
FROM table  
ORDER BY name2, id LIMIT 10000

3）SQLPrepare with

SELECT table.alotofcolumns, l01.languagedescription
FROM fetchcur('foo', ?, ?) table (column definitions) 
LEFT OUTER JOIN languagetable l01 ON (l01.lang = 'EN'
                                  AND l01.type = 'some type'
                                  AND l01.grp  = 'some group'
                                  AND l01.key  = table.somecolumn)
[~20 more LEFT OUTER JOINS in the same style, but for an other column]

4）SQLExecute，param1设置为SQL_FETCH_RELATIVE，param2设置为1
5）SQLExecute，param1设置为SQL_FETCH_RELATIVE，param2设置为-1
6）SQLExecute，param1设置为SQL_FETCH_RELATIVE，param2设置为0
7）解除所有，关闭游标，结束交易

函数fetchcur执行FETCH RELATIVE $3 IN $1 INTO rec，其中rec是一条记录并返回该记录。在用户请求时一次又一次地执行步骤4-6，同时在该事务中执行了更多的查询。在另一个用户请求发出之前还需要一段时间。通常，查询需要很长时间：

4）~130 ms
5）~115 ms
6）~110 ms

这对于快速用户体验来说通常太慢。所以我尝试使用psql命令行中的相同语句和\ timing on。对于步骤3-6，我使用了这些陈述：

3）

PREPARE p_foo (INTEGER, INTEGER) AS 
SELECT table.alotofcolumns, l01.languagedescription
FROM fetchcur('foo', $1, $2) table (column definitions) 
LEFT OUTER JOIN languagetable l01 ON (l01.lang = 'EN'
                                  AND l01.type = 'some type'
                                  AND l01.grp  = 'some group'
                                  AND l01.key  = table.somecolumn)
[~20 more LEFT OUTER JOINS in the same style, but for an other column]

4-6）

EXPLAIN ANALYZE EXECUTE p_foo (6, x);

对于第一次执行它花了89毫秒，但随后它下降到~7毫秒。即使我在执行之间等待几分钟，它仍然保持在每个查询不到10毫秒。那么额外的100毫秒哪里可以消失？应用程序和数据库位于同一系统上，因此网络延迟不应成为问题。每个LEFT OUTER JOIN仅返回一行，并且该结果中只有一列被添加到结果集中。所以结果是一行有~200列，大多数是VARCHAR和INTEGER类型。但这不应该是在同一台机器上传输大约100毫秒的数据。所以任何提示都会有所帮助。

机器有2 GB的RAM，参数设置为：

shared_buffers = 512MB
effective_cache_size = 256MB
work_mem = 16MB
maintenance_work_mem = 256MB
temp_buffers = 8MB
wal_buffers= 1MB

编辑：我刚刚发现如何从psqlodbc创建一个mylog，但我找不到时间值。

EDIT2：还可以为每一行添加时间戳。这确实表明，在psqlodbc驱动程序收到响应之前，它需要> 100ms。所以我再次尝试psql并添加选项-h 127.0.0.1以确保它也通过TCP / IP。 psql的结果是<10毫秒。这怎么可能？

00:07:51.026 [3086550720][SQLExecute]
00:07:51.026 [3086550720]PGAPI_Execute: entering...1
00:07:51.026 [3086550720]PGAPI_Execute: clear errors...
00:07:51.026 [3086550720]prepareParameters was not called, prepare state:3
00:07:51.026 [3086550720]SC_recycle_statement: self= 0x943b1e8
00:07:51.026 [3086550720]PDATA_free_params:  ENTER, self=0x943b38c
00:07:51.026 [3086550720]PDATA_free_params:  EXIT
00:07:51.026 [3086550720]Exec_with_parameters_resolved: copying statement params: trans_status=6, len=10128, stmt='SELECT [..]'
00:07:51.026 [3086550720]ResolveOneParam: from(fcType)=-15, to(fSqlType)=4(23)
00:07:51.026 [3086550720]cvt_null_date_string=0 pgtype=23 buf=(nil)
00:07:51.026 [3086550720]ResolveOneParam: from(fcType)=4, to(fSqlType)=4(23)
00:07:51.026 [3086550720]cvt_null_date_string=0 pgtype=23 buf=(nil)
00:07:51.026 [3086550720]   stmt_with_params = 'SELECT [..]'
00:07:51.027 [3086550720]about to begin SC_execute
00:07:51.027 [3086550720]       Sending SELECT statement on stmt=0x943b1e8, cursor_name='SQL_CUR0x943b1e8' qflag=0,1
00:07:51.027 [3086550720]CC_send_query: conn=0x9424668, query='SELECT [..]'
00:07:51.027 [3086550720]CC_send_query: conn=0x9424668, query='SAVEPOINT _EXEC_SVP_0x943b1e8'
00:07:51.027 [3086550720]send_query: done sending query 35bytes flushed
00:07:51.027 [3086550720]in QR_Constructor
00:07:51.027 [3086550720]exit QR_Constructor
00:07:51.027 [3086550720]read 21, global_socket_buffersize=4096
00:07:51.027 [3086550720]send_query: got id = 'C'
00:07:51.027 [3086550720]send_query: ok - 'C' - SAVEPOINT
00:07:51.027 [3086550720]send_query: setting cmdbuffer = 'SAVEPOINT'
00:07:51.027 [3086550720]send_query: returning res = 0x8781c90
00:07:51.027 [3086550720]send_query: got id = 'Z'
00:07:51.027 [3086550720]QResult: enter DESTRUCTOR
00:07:51.027 [3086550720]QResult: in QR_close_result
00:07:51.027 [3086550720]QResult: free memory in, fcount=0
00:07:51.027 [3086550720]QResult: free memory out
00:07:51.027 [3086550720]QResult: enter DESTRUCTOR
00:07:51.027 [3086550720]QResult: exit close_result
00:07:51.027 [3086550720]QResult: exit DESTRUCTOR
00:07:51.027 [3086550720]send_query: done sending query 1942bytes flushed
00:07:51.027 [3086550720]in QR_Constructor
00:07:51.027 [3086550720]exit QR_Constructor
00:07:51.027 [3086550720]read -1, global_socket_buffersize=4096
00:07:51.027 [3086550720]Lasterror=11
00:07:51.133 [3086550720]!!!  poll ret=1 revents=1
00:07:51.133 [3086550720]read 4096, global_socket_buffersize=4096
00:07:51.133 [3086550720]send_query: got id = 'T'
00:07:51.133 [3086550720]QR_fetch_tuples: cursor = '', self->cursor=(nil)
00:07:51.133 [3086550720]num_fields = 166
00:07:51.133 [3086550720]READING ATTTYPMOD
00:07:51.133 [3086550720]CI_read_fields: fieldname='id', adtid=23, adtsize=4, atttypmod=-1 (rel,att)=(0,0)
[last two lines repeated for the other columns]
00:07:51.138 [3086550720]QR_fetch_tuples: past CI_read_fields: num_fields = 166
00:07:51.138 [3086550720]MALLOC: tuple_size = 100, size = 132800
00:07:51.138 [3086550720]QR_next_tuple: inTuples = true, falling through: fcount = 0, fetch_number = 0
00:07:51.139 [3086550720]qresult: len=3, buffer='282'
[last line repeated for the other columns]
00:07:51.140 [3086550720]end of tuple list -- setting inUse to false: this = 0x87807e8 SELECT 1
00:07:51.140 [3086550720]_QR_next_tuple: 'C' fetch_total = 1 & this_fetch = 1
00:07:51.140 [3086550720]QR_next_tuple: backend_rows < CACHE_SIZE: brows = 0, cache_size = 0
00:07:51.140 [3086550720]QR_next_tuple: reached eof now
00:07:51.140 [3086550720]send_query: got id = 'Z'
00:07:51.140 [3086550720]     done sending the query:
00:07:51.140 [3086550720]extend_column_bindings: entering ... self=0x943b270, bindings_allocated=166, num_columns=166
00:07:51.140 [3086550720]exit extend_column_bindings=0x9469500
00:07:51.140 [3086550720]SC_set_Result(943b1e8, 87807e8)
00:07:51.140 [3086550720]QResult: enter DESTRUCTOR
00:07:51.140 [3086550720]retval=0
00:07:51.140 [3086550720]CC_send_query: conn=0x9424668, query='RELEASE _EXEC_SVP_0x943b1e8'
00:07:51.140 [3086550720]send_query: done sending query 33bytes flushed
00:07:51.140 [3086550720]in QR_Constructor
00:07:51.140 [3086550720]exit QR_Constructor
00:07:51.140 [3086550720]read -1, global_socket_buffersize=4096
00:07:51.140 [3086550720]Lasterror=11
00:07:51.140 [3086550720]!!!  poll ret=1 revents=1
00:07:51.140 [3086550720]read 19, global_socket_buffersize=4096
00:07:51.140 [3086550720]send_query: got id = 'C'
00:07:51.140 [3086550720]send_query: ok - 'C' - RELEASE
00:07:51.140 [3086550720]send_query: setting cmdbuffer = 'RELEASE'
00:07:51.140 [3086550720]send_query: returning res = 0x877cd30
00:07:51.140 [3086550720]send_query: got id = 'Z'
00:07:51.140 [3086550720]QResult: enter DESTRUCTOR
00:07:51.140 [3086550720]QResult: in QR_close_result
00:07:51.140 [3086550720]QResult: free memory in, fcount=0
00:07:51.140 [3086550720]QResult: free memory out
00:07:51.140 [3086550720]QResult: enter DESTRUCTOR
00:07:51.140 [3086550720]QResult: exit close_result
00:07:51.140 [3086550720]QResult: exit DESTRUCTOR

EDIT3：我意识到之前我没有在psql测试中使用mylog中的相同查询。似乎psqlodbc不对SQLPrepare和SQLExecute使用PREPARE。它添加了param值并发送查询。正如araqnid建议我将log_duration param设置为0并将postgresql日志的结果与app和psql的结果进行比较。结果如下：

                            psql/app     pglog
query executed from app:      110 ms     70 ms
psql with PREPARE/EXECUTE:     10 ms      5 ms
psql with full SELECT:         85 ms     70 ms

那么如何解释这些价值呢？似乎最多的时间是将完整的查询发送到数据库（10000个字符）并生成执行计划。如果确实如此，将对SQLPrepare和SQLExecute的调用更改为通过SQLExecDirect执行的显式PREPARE / EXECUTE语句可以解决问题。有异议吗？

Answer 1

我终于找到了问题，默认情况下psqlodbc的SQLPrepare / SQLExecute不执行PREPARE / EXECUTE。驱动程序本身构建SELECT并发送它。

解决方案是将UseServerSidePrepare=1添加到odbc.ini或添加到SQLDriverConnect的ConnectionString。从应用程序测量的一个查询的总执行时间从> 100ms降至5-10ms。

Answer 2

我不认为psql和你的程序之间的时间是可比的。

也许我错过了一些东西，但在psql中你只是准备语句，但从来没有真正获取数据。 EXPLAIN PLAN无法发送数据

因此时差很可能是将所有行从服务器发送到客户端所需的网络流量。

减少此时间的唯一方法是获得更快的网络或选择更少的列。您真的需要“alotofcolumns”中包含的所有列吗？

Postgres查询ODBC的速度慢了几个数量级？

2 个答案: