Question

我按照示例here索引目录中的所有pdf。这个过程似乎运行良好，但最后，当我进入Solr-UI并单击“执行查询”（使用q = *：*）时，我只得到一个条目。

我的代码中是否遗漏了某些内容？

...
String[] files = documentDir.list();

if (files != null)
{
  for (String document : files)
  {       
    ContentHandler textHandler = new BodyContentHandler();
    Metadata metadata = new Metadata();
    ParseContext context = new ParseContext();
    AutoDetectParser autoDetectParser = new AutoDetectParser();

    InputStream inputStream = null;

    try
    {
      inputStream = new FileInputStream(new File(documentDir, document));

      autoDetectParser.parse(inputStream, textHandler, metadata, context);

      SolrInputDocument doc = new SolrInputDocument();
      doc.addField("id", document);

      String content = textHandler.toString();

      if (content != null)
      {
        doc.addField("fullText", content);
      }

      UpdateResponse resp = server.add(doc, 1);

      server.commit(true, true, true);

      if (resp.getStatus() != 0)
      {
        throw new IDSystemException(LOG, "Document could not be indexed. Status returned: " + resp.getStatus());
      }
    }
    catch (FileNotFoundException fnfe)
    {
      throw new IDSystemException(LOG, fnfe.getMessage(), fnfe);
    }
    catch (IOException ioe)
    {
      throw new IDSystemException(LOG, ioe.getMessage(), ioe);
    }
    catch (SAXException se)
    {
      throw new IDSystemException(LOG, se.getMessage(), se);
    }
    catch (TikaException te)
    {
      throw new IDSystemException(LOG, te.getMessage(), te);
    }
    catch (SolrServerException sse)
    {
      throw new IDSystemException(LOG, sse.getMessage(), sse);
    }
    finally
    {
      if (inputStream != null)
      {
        try
        {
          inputStream.close();
        }
        catch (IOException ioe)
        {
          throw new IDSystemException(LOG, ioe.getMessage(), ioe);
        }
      }
    }
    ...

Answer 1

我有＆＃34; signatureField＆＃34;绑定到＆＃34; uid＆＃34;在solrconfig.xml中，所以uid始终是相同的。现在我为＆＃34; signatureField＆＃34;定义了一个新字段。它有效！

在：

...
<updateRequestProcessorChain name="deduplication">
    <processor
        class="org.apache.solr.update.processor.SignatureUpdateProcessorFactory">
        <bool name="overwriteDupes">false</bool>
        <str name="signatureField">uid</str>  <---------------------
        <bool name="enabled">true</bool>
        <str name="fields">content</str>
        <str name="minTokenLen">10</str>
        <str name="quantRate">.2</str>
        <str name="signatureClass">solr.update.processor.TextProfileSignature</str>
    </processor>
    <processor class="solr.LogUpdateProcessorFactory" />
    <processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
...


...
<fields>
    <field name="uid" type="string" indexed="true" stored="true" multiValued="false" />
    <dynamicField name="ignored_*" type="ignored" multiValued="true" indexed="false" stored="fasle" />
    <field name="id" type="string" indexed="true" stored="true" multiValued="false" />
    <field name="fullText" indexed="true" type="text" multiValued="true" />
</fields>
<uniqueKey>uid</uniqueKey>
...

后：

...

<updateRequestProcessorChain name="deduplication">
    <processor
        class="org.apache.solr.update.processor.SignatureUpdateProcessorFactory">
        <bool name="overwriteDupes">false</bool>
        <str name="signatureField">signatureField</str>  <---------------------
        <bool name="enabled">true</bool>
        <str name="fields">content</str>
        <str name="minTokenLen">10</str>
        <str name="quantRate">.2</str>
        <str name="signatureClass">solr.update.processor.TextProfileSignature</str>
    </processor>
    <processor class="solr.LogUpdateProcessorFactory" />
    <processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
...


...
<fields>
    <field name="uid" type="string" indexed="true" stored="true" multiValued="false" />
    <field name="signatureField" type="string" indexed="true" stored="true" multiValued="false" />  <----------------------------------
    <dynamicField name="ignored_*" type="ignored" multiValued="true" indexed="false" stored="fasle" />
    <field name="id" type="string" indexed="true" stored="true" multiValued="false" />
    <field name="fullText" indexed="true" type="text" multiValued="true" />
</fields>
<uniqueKey>uid</uniqueKey>
...

Answer 2

也许用server.add检查你的行。为什么你有1作为第二个参数？

您是否考虑过使用Nutch，它会为您执行文件抓取和索引编制吗？

许多PDF索引，但只有一个在Solr-UI中返回

2 个答案: