我们发现使用datanucleus和postgresql jdbc连接到远程postgres数据库的性能非常糟糕。原因是生成了许多查询数据类型表的查询:pg_catalog.pg_type。
我们写了一个小程序如下,并添加了datanucleus,JDO& postgres jdbc jars to classpath,它将在创建持久性管理器时从pg_catalog.pg_type获取每个单独的条目。
测试程序:
Map properties = new HashMap();
properties.put("javax.jdo.PersistenceManagerFactoryClass", "org.datanucleus.jdo.JDOPersistenceManagerFactory");
properties.put("datanucleus.storeManagerType", "rdbms");
properties.put("datanucleus.ConnectionDriverName", "org.postgresql.Driver");
properties.put("datanucleus.ConnectionURL", "jdbc:postgresql://xxxx:5432/mydb");
properties.put("datanucleus.ConnectionUserName", "xxx");
properties.put("datanucleus.ConnectionPassword", "xxx");
properties.put("datanucleus.autoCreateSchema", "false");
properties.put("datanucleus.autoCreateTables", "false");
properties.put("datanucleus.autoCreateColumns", "false");
properties.put("datanucleus.autoCreateConstraints", "false");
properties.put("datanucleus.validateSchema", "false");
properties.put("datanucleus.validateTables", "false");
properties.put("datanucleus.validateConstraints", "false");
properties.put("datanucleus.validateColumns", "false");
properties.put("datanucleus.metadata.validate", "false");
PersistenceManagerFactory pmf = JDOHelper.getPersistenceManagerFactory(properties);
PersistenceManager pm = PersistenceManagerFactoryUtils.getPersistenceManager(pmf, true);
Query query = pm.newQuery("javax.jdo.query.SQL", "select * from dblanguage limit 10");
Object result = query.execute();
System.out.println("completed");
以下是我们在数据库日志文件中找到的已执行查询的一部分:
51 HKT LOG: execute <unnamed>: SELECT t.typlen FROM pg_catalog.pg_type t, pg_catalog.pg_namespace n WHERE t.typnamespace=n.oid AND t.typname='name' AND n.nspname='pg_catalog'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT t.typname,t.oid FROM pg_catalog.pg_type t JOIN pg_catalog.pg_namespace n ON (t.typnamespace = n.oid) WHERE n.nspname != 'pg_toast'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT 1 FROM pg_catalog.pg_type WHERE typname = $1 AND typinput='array_in'::regproc
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = 'int2vector'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT typname FROM pg_catalog.pg_type WHERE oid = $1
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = '22'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT 1 FROM pg_catalog.pg_type WHERE typname = $1 AND typinput='array_in'::regproc
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = 'regproc'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT typname FROM pg_catalog.pg_type WHERE oid = $1
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = '24'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT 1 FROM pg_catalog.pg_type WHERE typname = $1 AND typinput='array_in'::regproc
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = 'tid'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT typname FROM pg_catalog.pg_type WHERE oid = $1
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = '27'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT 1 FROM pg_catalog.pg_type WHERE typname = $1 AND typinput='array_in'::regproc
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = 'xid'
2011-11-18 15:44:51 HKT LOG: execute <unnamed>: SELECT typname FROM pg_catalog.pg_type WHERE oid = $1
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = '28'
2011-11-18 15:44:51 HKT LOG: execute S_1: SELECT 1 FROM pg_catalog.pg_type WHERE typname = $1 AND typinput='array_in'::regproc
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = 'cid'
2011-11-18 15:44:51 HKT LOG: execute S_2: SELECT typname FROM pg_catalog.pg_type WHERE oid = $1
2011-11-18 15:44:51 HKT DETAIL: parameters: $1 = '29'
......
由于执行的查询数量太多(数千个),因此会降低性能。有谁知道如何解决这个问题?感谢。
在我进一步跟踪到datanucleus库之后,这里是stacktrace,显示它如何调用从rdbms检索数据类型信息:
RDBMSSchemaHandler.getRDBMSTypesInfo(Connection) line: 359
RDBMSSchemaHandler.getSchemaData(Object, String, Object[]) line: 167
PostgreSQLAdapter(DatabaseAdapter).initialiseTypes(StoreSchemaHandler, ManagedConnection) line: 462
PostgreSQLAdapter.initialiseTypes(StoreSchemaHandler, ManagedConnection) line: 119
RDBMSStoreManager.<init>(ClassLoaderResolver, OMFContext) line: 304
NativeConstructorAccessorImpl.newInstance0(Constructor, Object[]) line: not available [native method]
NativeConstructorAccessorImpl.newInstance(Object[]) line: not available
DelegatingConstructorAccessorImpl.newInstance(Object[]) line: not available
Constructor<T>.newInstance(Object...) line: not available
NonManagedPluginRegistry.createExecutableExtension(ConfigurationElement, String, Class[], Object[]) line: 587
PluginManager.createExecutableExtension(String, String, String, String, Class[], Object[]) line: 300
FederationManager.initialiseStoreManager(ClassLoaderResolver) line: 173
FederationManager.<init>(ClassLoaderResolver, OMFContext) line: 74
JDOPersistenceManagerFactory(ObjectManagerFactoryImpl).initialiseStoreManager(ClassLoaderResolver) line: 139
JDOPersistenceManagerFactory.freezeConfiguration() line: 583
JDOPersistenceManagerFactory.createPersistenceManagerFactory(Map) line: 286
JDOPersistenceManagerFactory.getPersistenceManagerFactory(Map) line: 182
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not available
Method.invoke(Object, Object...) line: not available
JDOHelper$16.run() line: 1958
AccessController.doPrivileged(PrivilegedExceptionAction<T>) line: not available [native method]
JDOHelper.invoke(Method, Object, Object[]) line: 1953
JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(String, Map<?,?>, Map<?,?>, ClassLoader) line: 1159
JDOHelper.getPersistenceManagerFactory(Map<?,?>, Map<?,?>, ClassLoader) line: 803
JDOHelper.getPersistenceManagerFactory(Map<?,?>) line: 698
PersistenceManagerFactoryTest.createPersistenceManagerFactory() line: 33
PersistenceManagerFactoryTest.main(String[]) line: 86