Spring Database Load Test:如何释放连接

时间:2016-12-19 08:53:29

标签: java mysql spring tomcat

我只是想在一个小型的Show-Case-Application中简化我的问题 似乎在Thread完成之前Connection不会关闭。 Check this out

依赖关系信息:

Servlet引擎:Apache Tomcat / 8.5.5
JPA提供商: Hibernate

我的pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>...</groupId>
    <artifactId>***-service</artifactId>
    <version>0.0.1</version>
    <packaging>jar</packaging>

    <name>...</name>
    <description>...</description>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
        <version>1.4.1.RELEASE</version>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <spring.cloud.release.version>Brixton.SR2</spring.cloud.release.version>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-thymeleaf</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-rest</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- eureka discovery client -->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-eureka</artifactId>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>

        <!-- for unit tests -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!-- for rest documentation -->
        <dependency>
            <groupId>org.springframework.restdocs</groupId>
            <artifactId>spring-restdocs-mockmvc</artifactId>
            <scope>test</scope>
        </dependency>

        <!-- for (un)marshalling of json / xml -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-xml</artifactId>
        </dependency>

        <dependency>
            <!-- jsoup HTML parser library @ http://jsoup.org/ -->
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.9.2</version>
        </dependency>

    </dependencies>

    <dependencyManagement>
        <dependencies>
            <!-- also needed for eureka discovery -->
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring.cloud.release.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <finalName>${artifactId}</finalName>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <executable>true</executable>
                    <outputDirectory>${dir}</outputDirectory>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

我有一个带MYSQL的Spring Boot Server作为DB。当我使用超过100个线程/用户进行负载测试(JMeter)时,我得到了这个例外:

  

org.apache.tomcat.jdbc.pool.PoolExhaustedException:[http-nio-8015-exec-195]超时:池为空。无法在60秒内获取连接,无法使用[size:50;忙:50;空闲:0; lastwait:60000]。

我的数据源配置:

spring.datasource.tomcat.minIdle = 0
spring.datasource.tomcat.maxIdle = 10
spring.datasource.tomcat.maxActive = 50
spring.datasource.tomcat.maxWait = 60000
spring.datasource.tomcat.testOnBorrow = true
spring.datasource.tomcat.timeBetweenEvictionRunsMillis = 1800000 // Updated
spring.datasource.tomcat.numTestsPerEvictionRun = 50  // Updated
spring.datasource.tomcat.minEvictableIdleTimeMillis = 10
spring.datasource.tomcat.validationQuery = SELECT 1
spring.datasource.tomcat.testWhileIdle = true

数据库上唯一的查询是简单的SELECT * FROM TABLE WHERE deletedat IS NULL。它在大约0.01秒内返回大约4000个条目(通过mysql CLI) 我已经安装了Neor Profile SQL来配置我的MYSQL DB。当我检查所有进程时,我的Schema上的进程都处于休眠状态。看起来我的配置不会释放那些连接。

我正在使用一个从CrudRepository扩展的简单接口:

public interface MyRepository extends CrudRepository<MyModel, String> {
    @Query("SELECT m from MyModel m WHERE m.deletedAt IS NULL")
    List<MyModel> findWhereDeletedAtIsNull();
}

并使用它

List<MyModel> myModelList = myRepository.findWhereDeletedAtIsNull();

我缺少什么?
似乎需要调整配置或代码才能释放连接。

Hibernate配置:

spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect  

2 个答案:

答案 0 :(得分:8)

您的jpa配置注册了OpenEntityManagerInViewInterceptor,它将JPA EntityManager绑定到工作线程以进行整个请求处理。因此,在完成服务方法(com.example.api.UserService.getAllUser())后,数据库连接将被关闭(或返回池)。

这会使您的数据库连接长时间保持打开状态,并且您的连接池在加载过程中会耗尽。

要解决您的问题,您可以通过在application.properties中添加below property来停用OpenEntityManagerInViewInterceptor过滤器的自动注册

  

spring.jpa.open-in-view = false

默认情况下,此属性为true。

查看相关问题:  1. Consider not registering OpenEntityManagerInViewInterceptor by default

答案 1 :(得分:1)

这里的关键是启用logAbandoned。这样,Tomcat的JDBC池将告诉您在哪里正在分配未被返回的连接。截至目前,您只是通过覆盖它来尝试从问题中恢复:强制池释放未正确返回的连接。

java.sql.SQLException: Connection has already been closed正在发生,因为您的查询完成时间超过50秒,但您的代码正在清理它。因此,50秒后,池关闭连接(因为它已被“放弃”),然后当您的代码获得控制权时,它也会关闭连接。考虑提高您的废弃连接超时。