HibernateSearch，JPA，H2驱动程序在数据库索引过程中抛出异常

Question

使用HibernateSearch我想索引我的H2嵌入式数据库。
调用此代码：

EntityManager em = articleDao.getEntityManager();
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(em);
try {
    fullTextEntityManager.createIndexer().progressMonitor(new CustomMassIndexerProcessMonitor()).startAndWait();
} catch (InterruptedException e) {
    e.printStackTrace();
}  

索引几分钟后会抛出异常：

2013-09-04 09:01:41 ERROR LogErrorHandler.handleException():83 - HSEARCH000058: HSEARCH000116: Unexpected error during MassIndexer operation
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuffer.append(Unknown Source)
    at java.io.StringWriter.write(Unknown Source)
    at org.h2.util.IOUtils.copyAndCloseInput(IOUtils.java:201)
    at org.h2.util.IOUtils.readStringAndClose(IOUtils.java:301)
    at org.h2.value.ValueLobDb.getString(ValueLobDb.java:226)
    at org.h2.jdbc.JdbcResultSet.getString(JdbcResultSet.java:296)
    at org.hibernate.type.descriptor.sql.VarcharTypeDescriptor$2.doExtract(VarcharTypeDescriptor.java:66)
    at org.hibernate.type.descriptor.sql.BasicExtractor.extract(BasicExtractor.java:64)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:261)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:257)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:247)
    at org.hibernate.type.AbstractStandardBasicType.hydrate(AbstractStandardBasicType.java:332)
    at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:2912)
    at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1673)
    at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1605)
    at org.hibernate.loader.Loader.getRow(Loader.java:1505)
    at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:713)
    at org.hibernate.loader.Loader.processResultSet(Loader.java:943)
    at org.hibernate.loader.Loader.doQuery(Loader.java:911)
    at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:342)
    at org.hibernate.loader.Loader.doList(Loader.java:2526)
    at org.hibernate.loader.Loader.doList(Loader.java:2512)
    at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2342)
    at org.hibernate.loader.Loader.list(Loader.java:2337)
    at org.hibernate.loader.criteria.CriteriaLoader.list(CriteriaLoader.java:124)
    at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1662)
    at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:374)
    at org.hibernate.search.batchindexing.impl.IdentifierConsumerEntityProducer.loadList(IdentifierConsumerEntityProducer.java:151)
Hibernate Search: entityloader-2, CustomMassIndexerProcessMonitor entitiesLoaded(10)
Hibernate Search: collectionsloader-2, CustomMassIndexerProcessMonitor documentsAdded(1)
Hibernate Search: collectionsloader-2, CustomMassIndexerProcessMonitor documentsBuilt(1)
Hibernate Search: collectionsloader-3, CustomMassIndexerProcessMonitor documentsAdded(1)
Hibernate Search: collectionsloader-3, CustomMassIndexerProcessMonitor documentsBuilt(1)
2013-09-04 09:01:47 ERROR LogErrorHandler.handleException():83 - HSEARCH000058: HSEARCH000116: Unexpected error during MassIndexer operation
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOfRange(Unknown Source)
    at java.lang.String.<init>(Unknown Source)
    at java.lang.StringBuffer.toString(Unknown Source)
    at java.io.StringWriter.toString(Unknown Source)
    at org.h2.util.IOUtils.readStringAndClose(IOUtils.java:302)
    at org.h2.value.ValueLobDb.getString(ValueLobDb.java:226)
    at org.h2.jdbc.JdbcResultSet.getString(JdbcResultSet.java:296)
    at org.hibernate.type.descriptor.sql.VarcharTypeDescriptor$2.doExtract(VarcharTypeDescriptor.java:66)
    at org.hibernate.type.descriptor.sql.BasicExtractor.extract(BasicExtractor.java:64)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:261)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:257)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:247)
    at org.hibernate.type.AbstractStandardBasicType.hydrate(AbstractStandardBasicType.java:332)
    at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:2912)
    at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1673)
    at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1605)
    at org.hibernate.loader.Loader.getRow(Loader.java:1505)
    at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:713)
    at org.hibernate.loader.Loader.processResultSet(Loader.java:943)
    at org.hibernate.loader.Loader.doQuery(Loader.java:911)
    at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:342)
    at org.hibernate.loader.Loader.doList(Loader.java:2526)
    at org.hibernate.loader.Loader.doList(Loader.java:2512)
    at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2342)
    at org.hibernate.loader.Loader.list(Loader.java:2337)
    at org.hibernate.loader.criteria.CriteriaLoader.list(CriteriaLoader.java:124)
    at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1662)
    at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:374)
    at org.hibernate.search.batchindexing.impl.IdentifierConsumerEntityProducer.loadList(IdentifierConsumerEntityProducer.java:151)
    at org.hibernate.search.batchindexing.impl.IdentifierConsumerEntityProducer.loadAllFromQueue(IdentifierConsumerEntityProducer.java:117)
    at org.hibernate.search.batchindexing.impl.IdentifierConsumerEntityProducer.run(IdentifierConsumerEntityProducer.java:94)
    at org.hibernate.search.batchindexing.impl.OptionallyWrapInJTATransaction.run(OptionallyWrapInJTATransaction.java:132)

似乎H2 util类中的一个在尝试从DB读取时抛出此异常。我试图使用'-Xms1024m -Xmx2048m'增加堆，但这没有帮助:(
该方案如下。我的H2数据库的每个条目都有一个字段类型CLOB。如果我将小内容写入此字段，那么一切都很好，不会抛出任何错误。 但是如果我在这些字段中有大量内容（每个900kb），则在索引过程中会抛出错误。

我正在使用以下罐子：
hibernate-entitymanager 4.2.4.Final
h2 1.3.173
hibernate-search 4.4.0.Alpha1

这是我的持久性单元配置：

<persistence-unit name="hibernateSearchH2TestPersistenceUnit" transaction-type="RESOURCE_LOCAL">
    <provider>org.hibernate.ejb.HibernatePersistence</provider>

    <mapping-file>META-INF/queriesForTest.xml</mapping-file>

    <class>com.kaidex.db.entity.DocStatus</class>
    <class>com.kaidex.db.entity.DocType</class>
    <class>com.kaidex.db.entity.Article</class>
    <class>com.kaidex.db.entity.Issuer</class>
    <class>com.kaidex.db.entity.PublishingSource</class>

    <properties>
        <property name="hibernate.connection.url" value="jdbc:h2:D:\\kaidextestdb;CIPHER=XTEA"/>

        <property name="hibernate.dialect" value="org.hibernate.dialect.H2Dialect"/>
        <property name="hibernate.connection.driver_class" value="org.h2.Driver"/>

        <property name="hibernate.connection.username" value="sa"/>
        <property name="hibernate.connection.password" value="filepass userpass"/>

        <property name="hibernate.format_sql" value="true"/>
        <property name="hibernate.show_sql" value="false" />
        <property name="hibernate.hbm2ddl.auto" value="update" />


        <property name="hibernate.search.default.directory_provider" value="filesystem"/> 
        <property name="hibernate.search.default.indexBase" value="D:\lucene"/>
        <property name="hibernate.search.lucene_version" value="LUCENE_36"/>
    </properties>
</persistence-unit>

已更新。已添加实体配置：

@Entity(name="Article")
@Table(name="Article", schema="Kaidexdb")
@Indexed
public class Article {
    @Id
    @GeneratedValue(strategy=GenerationType.IDENTITY)
    private long id;
    ... 
    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    @Column(columnDefinition="CLOB")
    private String contentRo;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    @Column(columnDefinition="CLOB")
    private String contentRu;

    @IndexedEmbedded
    @ManyToOne
    @JoinColumn(name="docType_id", nullable=false)  
    private DocType docType;

    @IndexedEmbedded
    @ManyToOne
    @JoinColumn(name="docStatus_id", nullable=false)    
    private DocStatus docStatus;

    @IndexedEmbedded
    @ManyToOne
    @JoinColumn(name="issuer_id", nullable=false)
    private Issuer issuer;

    @IndexedEmbedded
    @ManyToOne
    @JoinColumn(name="ps_id", nullable=false)
    private PublishingSource publishingSource;
...


@Entity(name="DocStatus")
@Table(name="DocStatus", schema="Kaidexdb")
@Indexed
public class DocStatus {
    @Id
    @GeneratedValue(strategy=GenerationType.IDENTITY)
    private Long id;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRo;
    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRu;

    @OneToMany(mappedBy="docStatus", targetEntity=Article.class)
    private List<Article> articles; 
...

@Entity(name="DocType")
@Table(name="DocType", schema="Kaidexdb")
@Indexed
public class DocType {

    @Id
    @GeneratedValue(strategy=GenerationType.IDENTITY)
    private long id;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    @Column(unique=true)
    private String shortName;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRo;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRu;

    @OneToMany(mappedBy="docType", targetEntity=Article.class)
    private List<Article> articles;
...


@Entity(name="Issuer")
@Table(name="Issuer", schema="Kaidexdb")
@Indexed
public class Issuer {
    @Id
    @GeneratedValue(strategy=GenerationType.IDENTITY)
    private long id;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String shortNameRo; 

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRo;  

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRu;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    @Column(name="parent_id")
    private long parentId; 

    @OneToMany(mappedBy="issuer", targetEntity=Article.class)
    private List<Article> articles;
...


@Entity
@Table(name="PublishingSource", schema="Kaidexdb")
@Indexed
public class PublishingSource {
    @Id
    @GeneratedValue(strategy=GenerationType.IDENTITY)
    private long id;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRo;

    @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
    private String longNameRu;


    @OneToMany(mappedBy="publishingSource", targetEntity=Article.class)
    private List<Article> articles; 

有人可以帮我解决这个问题吗？
也许我应该对我的H2嵌入式数据库做一些特定的配置，告知H2我使用了大的CLOB字段？

提前谢谢。

Answer 1

根据堆栈跟踪，Hibernate搜索尝试从数据库中加载CLOB作为String（使用java.sql.ResultSet.getString）。因此，H2必须完全加载CLOB。此外，看起来Hibernate Search在内存中保留了如此大的字符串列表：

at org.h2.value.ValueLobDb.getString(ValueLobDb.java:226)
at org.h2.jdbc.JdbcResultSet.getString(JdbcResultSet.java:296)
at org.hibernate.type.descriptor.sql.VarcharTypeDescriptor$2.doExtract(VarcharTypeDescriptor.java:66)
...
at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:374)
at org.hibernate.search.batchindexing.impl.IdentifierConsumerEntityProducer.loadList(IdentifierConsumerEntityProducer.java:151)

所以它看起来像是Hibernate Search中的一个问题。与Hibernate Search have been reported before相关的内存问题（我知道这是旧版本），但我首先尝试的是使用不同版本的Hibernate Search，特别是因为你使用的是Alpha版本（4.4.0） .Alpha1）。可能是已知的问题。

Answer 2

我已经部分地解决了这个问题。我减少了每个查询的Hibernate Search加载的对象数量：

EntityManager em = articleDao.getEntityManager();
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(em);
fullTextEntityManager.createIndexer().batchSizeToLoadObjects(2).startAndWait();

如果H2开发人员在下一版本中修复此问题，那将会很棒 我已经使用HSQLDB和Apache Derby测试了相同的代码，并且那些db驱动程序没有抛出任何异常。