(编辑:ubuntu上的solr-6.6.0)
我试图使用Solr的DataImportHandler来索引MySQL数据库,其中包括带有RTF文件的blob条目。为此,我使用 FieldStreamDataSource ,如以下答案中所述:
How do I index Rich Format Documents in Blobs
Unsupported type Exception on Importing Documents from Database
当所有其他非Blob字段被索引时,我不断从FieldStreamDataSource.getData()方法获得以下Java 运行时错误,如Solr日志文件中所示:
引起:java.lang.RuntimeException:不支持的类型:class java.lang.String at org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:77) 在 org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:47) 在 org.apache.solr.handler.dataimport.DebugLogger $ 2.getData(DebugLogger.java:187)
其中涉及Java class的这些行(请参阅箭头'< ==='):
@Override
public InputStream getData(String query) {
Object o = wrapper.getVariableResolver().resolve(dataField);
if (o == null) {
throw new DataImportHandlerException(SEVERE, "No field available for name : " + dataField);
} else if (o instanceof Blob) { // <========= XXX
Blob blob = (Blob) o;
try {
return blob.getBinaryStream();
} catch (SQLException sqle) {
LOG.info("Unable to get data from BLOB");
return null;
}
} else if (o instanceof byte[]) {
byte[] bytes = (byte[]) o;
return new ByteArrayInputStream(bytes);
} else {
throw new RuntimeException("unsupported type : " + o.getClass()); // <========= XXX
}
这应该意味着getData()方法没有获得Blob类型,而是获取String。
我知道在许多其他线程中已经建议在数据库中blob值为null时出现此错误:
Unsupported type Exception on Importing Documents from Database
Solr DIH Throwing Error Unsupported Type Class Java.Lang.String
然而,这不是这种情况,因为我在这个测试数据库中只有5个条目,其中没有一个具有空值。 另外,我正在检查来自Solr DIH接口的调试输出,这表明确实从数据库中检索了Blob值(参见箭头&#39;&lt; ===&#39;) :
"verbose-output": [
"entity:reports1",
[
"document#1",
[
"query",
"SELECT id, institute, exam_date, age, acc, pacs FROM reports",
"time-taken",
"0:0:0.8",
null,
"----------- row #1-------------",
"id",
"1",
"institute",
"RADIOLOGY",
"age",
"68",
"acc",
"165184654",
"pacs",
"233215",
"exame_date",
"2016-02-05T00:00:00Z",
null,
"---------------------------------------------",
"entity:reports2",
[
"query",
"SELECT report FROM reports WHERE id='1'",
"time-taken",
"0:0:0.6",
null,
"----------- row #1-------------",
"report", // <========= COLUMN NAME RETURNED FROM THE SQL SELECT
"e1xydGYxXGFkZWZ[...]", // <========= VALUE RETURNED FROM THE SQL SELECT
null,
"---------------------------------------------",
"entity:report",
[
"query",
"report",
"EXCEPTION", // <========== EXCEPTION THROWN
"java.lang.RuntimeException: unsupported type : class java.lang.String\n\tat org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:77)\n\tat org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:47)\n\tat org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:187)\n\tat org.apache.solr.handler.dataimport.TikaEntityProcessor.nextRow(TikaEntityProcessor.java:128)\n\tat org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:267)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:516)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:516)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)\n\tat org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)\n\tat org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)\n\tat org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:415)\n\tat org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:474)\n\tat org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:180)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat java.lang.Thread.run(Thread.java:748)\n",
"time-taken",
"0:0:0.0"
]
]
],
这是我的 data-config.xml
<dataConfig>
<dataSource
name="db"
type="JdbcDataSource"
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/RIS"
user="root"
password="********"/>
<dataSource name="fieldStream" type="FieldStreamDataSource"/>
<document>
<entity
name="reports1"
query="SELECT id, institute, exam_date, age, acc, pacs FROM reports"
dataSource="db"
>
<field column="id" name="id"/>
<field column="institute" name="institute"/>
<field column="exam_date" name="exam_date"/>
<field column="age" name="age"/>
<field column="acc" name="acc"/>
<field column="pacs" name="pacs"/>
<entity
name="reports2"
query="SELECT report FROM reports WHERE id='${reports1.id}'"
dataSource="db"
>
<entity
name="report"
dataSource="fieldStream"
processor="TikaEntityProcessor"
url="report"
dataField="reports2.REPORT"
format="text"
onError="continue">
<field column="text" name="report"/>
</entity>
</entity>
</entity>
</document>
</dataConfig>
所以在我看来,从数据库的blob列中检索到的值正在传递或重新计算为字符串,而不是Blob类型。任何人都可以帮我看看我是否&#39我做错了什么?我到处搜索,无法看到解决方案:/
非常感谢
答案 0 :(得分:0)
我找到了(相当微不足道的)问题(即使我已经尝试了几天)
将dataField属性更改为小写就行了,这与this回答的经验相反。将以下更改(请参阅箭头&#39;&lt; ===&#39;)更改为data-config.xml,并将所有条目编入索引。
<entity
name="reports2"
query="SELECT report FROM reports WHERE id='${reports1.id}'"
dataSource="db">
<entity
name="report"
dataSource="fieldStream"
processor="TikaEntityProcessor"
url="report"
dataField="reports2.report" // <===== instead of reports2.REPORT
format="text"
onError="continue">
<field column="text" name="report"/>
</entity>
然而,这确实给了我一个不相关的问题,可能值得指出(不确定是否在另一个答案中)。
因为从数据库返回的列(&#34; report&#34;)与Solr Schema字段具有相同的名称(&#34; report&#34;),所以索引在SQL查询,索引之后自动发生来自BLOB的二进制值。它忽略了TikaEntityProcessor在以下实体中提取的文本。我通过确保两个字段(MySQL和Solr)具有不同的名称来修复此问题。
下面,箭头(&#39;&lt; ===&#39;)显示索引发生的位置(早于计划)。
<entity
name="reports2"
query="SELECT report FROM reports WHERE id='${reports1.id}'" // <======== DIH indexed this column (report) from the DB into the Solr field of same name (binary representation)
dataSource="db">
<entity
name="report"
dataSource="fieldStream"
processor="TikaEntityProcessor"
url="report"
dataField="reports2.report"
format="text"
onError="continue">
<field column="text" name="report"/> //<======== instead of waiting for this column (text), the output from Tika (extracted text)
</entity>
</entity>