Question

我是luene的新手，我想将索引文件保存到数据库中，但我有这个例子，我不会更改max_allowed_packet，但我想限制文件的大小。

Exception in thread "Lucene Merge Thread #0" org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.jdbc.JdbcStoreException: Failed to execute sql [insert into search_lucene (name_, value_, size_, lf_, deleted_) values ( ?, ?, ?, current_timestamp, ? )]; nested exception is com.mysql.jdbc.PacketTooBigException: Packet for query is too large (1286944 > 1048576). You can change this value on the server by setting the max_allowed_packet' variable. 
        at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309) 
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286) 
Caused by: org.apache.lucene.store.jdbc.JdbcStoreException: Failed to execute sql [insert into search_lucene (name_, value_, size_, lf_, deleted_) values ( ?, ?, ?, current_timestamp, ? )]; nested exception is com.mysql.jdbc.PacketTooBigException: Packet for query is too large (1286944 > 1048576). You can change this value on the server by setting the max_allowed_packet' variable. 
        at org.apache.lucene.store.jdbc.support.JdbcTemplate.executeUpdate(JdbcTemplate.java:185) 
        at org.apache.lucene.store.jdbc.index.AbstractJdbcIndexOutput.close(AbstractJdbcIndexOutput.java:47) 
        at org.apache.lucene.store.jdbc.index.RAMAndFileJdbcIndexOutput.close(RAMAndFileJdbcIndexOutput.java:81) 
        at org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:203) 
        at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:204) 
        at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4263) 
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884) 
        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205) 
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260) 
Caused by: com.mysql.jdbc.PacketTooBigException: Packet for query is too large (1286944 > 1048576). You can change this value on the server by setting the max_allowed_packet' variable. 
        at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3915) 
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2598) 
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778) 
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2825) 
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2156) 
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2459) 
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2376) 
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2360) 
        at org.apache.lucene.store.jdbc.support.JdbcTemplate.executeUpdate(JdbcTemplate.java:175)

Answer 1

从http://wiki.apache.org/lucene-java/LuceneFAQ#Can_I_store_the_Lucene_index_in_a_relational_database.3F

读取

我可以将Lucene索引存储在关系数据库中吗？

Lucene不支持开箱即用的功能，但有几个人已经实现了JdbcDirectory。到目前为止我们看到的报告表明 这些实施的效果不是很好 ，但它是可行的。

Answer 2

要限制文件大小，您需要自己实现目录。诀窍是将每个文件分成几部分。也许您可以借用lucene-appengine中的一些代码，这些代码会将文件拆分为多个SegmentHunk。

我希望你知道你在做什么，因为在数据库中保留lucene索引比使用通常的内存映射文件慢得多。

Answer 3

essaies：

indexWriter.setUseCompoundFile(false);

LogByteSizeMergePolicy aLogByteSizeMergePolicy = new LogByteSizeMergePolicy();
aLogByteSizeMergePolicy.setMaxMergeMB(2);
aLogByteSizeMergePolicy.setMaxMergeMBForForcedMerge(4);
aLogByteSizeMergePolicy.setUseCompoundFile(false);
//voir aussi setMaxCFSSegmentSizeMB, setMaxMergeDocs, setMergeFactor

indexWriter.setMergePolicy(aLogByteSizeMergePolicy);
//Deprecated. use IndexWriterConfig.setMergePolicy(MergePolicy) instead.

Lucene在保存到数据库时限制文件大小

3 个答案: