我试图启用hibernate批量插入和更新,但无论我在哪里放置参数,它们似乎都没有生效。我的Spring配置主要只是XML。我们正在使用Spring 4.3.4
,Hibernate 5.2.5
,Spring Data 1.10.5
和MySQL Connector 5.1.38
我的MySql连接字符串是jdbc:mysql://localhost:3306/DB_NAME_HERE?autoReconnect=true&useTimezone=false&rewriteBatchedStatements=true
我的jpaContext.xml中有这个(我省略了整个事情,只包括我设置参数的部分)
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager" primary="true">
<property name="entityManagerFactory" ref="entityManagerFactory" />
<property name="jpaProperties">
<props>
<prop key="hibernate.jdbc.batch_size">100</prop>
<prop key="hibernate.order_inserts">true</prop>
<prop key="hibernate.order_updates" >true</prop>
<prop key="hibernate.jdbc.fetch_size" >400</prop>
<prop key="hibernate.jdbc.batch_versioned_data" >true</prop>
</props>
</property>
</bean>
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceUnitName" value="JpaPersistenceUnit" />
<property name="dataSource" ref="dataSource" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="false"/>
<property name="generateDdl" value="false"/>
<property name="database" value="MYSQL"/>
</bean>
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.jdbc.batch_size">100</prop>
<prop key="hibernate.order_inserts">true</prop>
<prop key="hibernate.order_updates" >true</prop>
<prop key="hibernate.jdbc.fetch_size" >400</prop>
<prop key="hibernate.jdbc.batch_versioned_data" >true</prop>
</props>
</property>
</bean>
我的persistance.xml看起来像这样
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="JpaPersistenceUnit">
<properties>
<property name="hibernate.session_factory.interceptor" value="com.siftit.startup.HibernateInterceptor" />
<property name="hibernate.jdbc.batch_size" value="100"/>
<property name="hibernate.order_inserts" value="true"/>
<property name="hibernate.order_updates" value="true"/>
<property name="hibernate.jdbc.fetch_size" value="400"/>
<property name="hibernate.jdbc.batch_versioned_data" value="true"/>
</properties>
</persistence-unit>
<persistence-unit name="warehousePersistenceUnit"/>
</persistence>
每个区域中重复参数的原因是我试图弄清楚它们属于哪里,但无论我把它们放在哪里,它们似乎都没有用。
这是配置的hibernate日志记录输出:
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - SessionFactory name : null
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Automatic flush during beforeCompletion(): enabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Automatic session close at end of transaction: disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Statistics: disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Deleted entity synthetic identifier rollback: disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Default entity-mode: pojo
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Check Nullability in Core (should be disabled when Bean Validation is on): enabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Allow initialization of lazy state outside session : disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Using BatchFetchStyle : LEGACY
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Default batch fetch size: -1
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Maximum outer join fetch depth: 2
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Default null ordering: NONE
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Order SQL updates by primary key: disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Order SQL inserts for batching: disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - multi-tenancy strategy : NONE
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - JTA Track by Thread: enabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Query language substitutions: {}
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - JPA query language strict compliance: disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Named query checking : enabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Second-level cache: enabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Second-level query cache: disabled
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Second-level query cache factory: org.hibernate.cache.internal.StandardQueryCacheFactory@2033240a
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Second-level cache region prefix: null
18:38:03,975 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Optimize second-level cache for minimal puts: disabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Structured second-level cache entries: disabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Second-level cache direct-reference entries: disabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Automatic eviction of collection cache: disabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - JDBC batch size: 15
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - JDBC batch updates for versioned data: enabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Scrollable result sets: enabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Wrap result sets: disabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - JDBC3 getGeneratedKeys(): enabled
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - JDBC result set fetch size: null
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Connection release mode: ON_CLOSE
18:38:03,991 [ContainerBackgroundProcessor[StandardEngine[Catalina]]] DEBUG org.hibernate.cfg.Settings: - Generate SQL with comments: disabled
到目前为止我尝试过的所有事情都失败了,似乎我必须做一些愚蠢的事情,因为其他人似乎都在工作,但到目前为止我还没有运气。 / p>
答案 0 :(得分:0)
如果您想要有效的插入和更新,至少需要做两件事。
确保所有实体都使用TableGenerated ID
@TableGenerator(name="USER_GENERATOR", table="SEQUENCE", pkColumnName="SEQ_NAME",
valueColumnName="SEQ_COUNT", pkColumnValue="USER_SEQ", allocationSize = 1)
@GeneratedValue(strategy = GenerationType.TABLE, generator = "USER_GENERATOR")
如果对id生成使用自动增量,则每个插入必须拥有SQL字符串,并且JPA需要在每次插入后选择LAST_INSERT_ID()
以获取自动增量ID。我的猜测是hibernate.jdbc.fetch_size
被驾驶员忽略了。
我没有尝试使用最新的MySQL JDBC驱动程序,但几年前,您需要通过在JDBC URL上包含rewriteBatchedStatements=true
来告诉驱动程序重写批处理语句。如果您不这样做,您可能会批量处理许多插入内容,但每个插入内容都包含所有列名称,您将无法获得Insert into Table (columns...) Values (values..., values..., )
修改强>
根据MySQL JDBC documentation,只有两个有效选项,将所有行读入前面的结果集,或者使用当前一行的游标读取行。我的猜测是hibernate.jdbc.fetch_size
被驾驶员忽略了。
答案 1 :(得分:0)
hibernate.jdbc.batch_size
将在Hibernate使用的基础JDBC连接上配置批处理。
除此之外,您还需要实现一些批处理和执行更新/插入的方法。
EntityManager em = getEntityManager();
int batchSize = 1000;
for (int i = 0; i < persons.size(); i++) {
Person person = persons.get(i);
em.persist(person);
if(i % batchSize == 0) {
em.flush();
em.clear();
}
}
em.flush();
em.clear();