使用Hive Metastore + HikariCP

时间:2020-07-31 12:20:29

标签: postgresql hive hikaricp hive-metastore

我将Presto与Hive Metastore结合使用,后者配置为使用HikariCP,并由Postgres数据库支持。

使用默认值,我观察到Metastore创建了2个Hikari连接池,每个连接池的maximumPoolSize设置为10。当一切启动后,我发现Postgres中有20个连接到 metastore 数据库的连接

[HikariPool-1 housekeeper] pool.HikariPool: HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
[HikariPool-2 housekeeper] pool.HikariPool: HikariPool-2 - Pool stats (total=10, active=0, idle=10, waiting=0)

但是,随着时间的流逝,随着查询的提交以及客户端连接到Metastore,我发现Postgres中的连接数超过了20。我期望池大小是固定的,后端数据库连接数不会超过20,即maximumPoolSize(10)x池数(2)。为什么会发生这种情况的任何见解?

例如

postgres=# SELECT COUNT(*) from pg_stat_activity where datname = 'metastore';

 count 
-------
    38
(1 row)


postgres=# SELECT COUNT(*) from pg_stat_activity where datname = 'metastore';
 count 
-------
    57

版本:

  • 蜂巢= 2.3.6
  • HikariCP = 2.5.1
  • DataNucleus = 4.1.17
  • Postgres服务器= 12

HikariConfig:

hikari.HikariConfig: HikariPool-1 - configuration:
hikari.HikariConfig: allowPoolSuspension.............false
hikari.HikariConfig: autoCommit......................true
hikari.HikariConfig: catalog.........................null
hikari.HikariConfig: connectionInitSql...............null
hikari.HikariConfig: connectionTestQuery.............null
hikari.HikariConfig: connectionTimeout...............30000
hikari.HikariConfig: dataSource......................null
hikari.HikariConfig: dataSourceClassName.............null
hikari.HikariConfig: dataSourceJNDI..................null
hikari.HikariConfig: dataSourceProperties............{password=<masked>}
hikari.HikariConfig: driverClassName................."org.postgresql.Driver"
hikari.HikariConfig: healthCheckProperties...........{}
hikari.HikariConfig: healthCheckRegistry.............null
hikari.HikariConfig: idleTimeout.....................600000
hikari.HikariConfig: initializationFailFast..........true
hikari.HikariConfig: isolateInternalQueries..........false
hikari.HikariConfig: jdbc4ConnectionTest.............false
hikari.HikariConfig: jdbcUrl........................."jdbc:postgresql://hive-db:5432/metastore"
hikari.HikariConfig: leakDetectionThreshold..........0
hikari.HikariConfig: maxLifetime.....................1800000
hikari.HikariConfig: maximumPoolSize.................10
hikari.HikariConfig: metricRegistry..................null
hikari.HikariConfig: metricsTrackerFactory...........null
hikari.HikariConfig: minimumIdle.....................10
hikari.HikariConfig: password........................<masked>
hikari.HikariConfig: poolName........................"HikariPool-1"
hikari.HikariConfig: readOnly........................false
hikari.HikariConfig: registerMbeans..................false
hikari.HikariConfig: scheduledExecutorService........null
hikari.HikariConfig: threadFactory...................null
hikari.HikariConfig: transactionIsolation............null
hikari.HikariConfig: username........................"postgres"
hikari.HikariConfig: validationTimeout...............5000

hikari.HikariConfig: HikariPool-2 - configuration:
hikari.HikariConfig: allowPoolSuspension.............false
hikari.HikariConfig: autoCommit......................true
hikari.HikariConfig: catalog.........................null
hikari.HikariConfig: connectionInitSql...............null
hikari.HikariConfig: connectionTestQuery.............null
hikari.HikariConfig: connectionTimeout...............30000
...
...

0 个答案:

没有答案