Question

当我运行以下命令时，Nutch索引失败：

root@ubuntu:/home/test-tb/Downloads/apache-nutch-1.10# bin/nutch index mycrl/crawldb/ -dir mycrl/segments/

我在ubuntu 12.04 LTS上使用nutch 1.10。

错误日志详细信息为：

2015-07-09 17:07:36,940 INFO  indexer.IndexWriters - Adding    org.apache.nutch.indexwriter.solr.SolrIndexWriter
2015-07-09 17:07:36,970 INFO  solr.SolrMappingReader - source: content dest: content
2015-07-09 17:07:36,970 INFO  solr.SolrMappingReader - source: title dest: title
2015-07-09 17:07:36,970 INFO  solr.SolrMappingReader - source: host dest: host
2015-07-09 17:07:36,970 INFO  solr.SolrMappingReader - source: segment dest: segment
2015-07-09 17:07:36,970 INFO  solr.SolrMappingReader - source: boost dest: boost
2015-07-09 17:07:36,970 INFO  solr.SolrMappingReader - source: digest dest: digest
2015-07-09 17:07:36,970 INFO  solr.SolrMappingReader - source: tstamp dest: tstamp
2015-07-09 17:07:37,030 INFO  solr.SolrIndexWriter - Indexing 100 documents
2015-07-09 17:07:37,136 INFO  solr.SolrIndexWriter - Indexing 100 documents
2015-07-09 17:07:37,166 WARN  mapred.LocalJobRunner - job_local1383488781_0001
org.apache.solr.common.SolrException: Not Found

Not Found

request: http://127.0.0.1:8983/solr/update?wt=javabin&version=2
at  org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:430)
at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.close(SolrIndexWriter.java:153)
at org.apache.nutch.indexer.IndexWriters.close(IndexWriters.java:115)
at org.apache.nutch.indexer.IndexerOutputFormat$1.close(IndexerOutputFormat.java:44)
at org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.close(ReduceTask.java:467)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:535)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:421)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:398)

2015-07-09 17:07:37,957 ERROR indexer.IndexingJob - Indexer: java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1357)
at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:113)
at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:177)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:187)

虽然我没有为nutch指定solr索引选项，但会返回此错误。我在这里错过了什么吗？你的指针将非常有帮助。提前谢谢。

Answer 1

首先，如果您要抓取数据并将其编入索引，那么您应该使用bin/crawl，因为它是一种更好的工具。

其次，从堆栈跟踪中可以清楚地看到，您没有正确设置solr url。通常，您的solr网址应该类似于http://domainname:port/solr/corename

但是，我看到你有localhost:8983/solr/update。所以，你的网址缺少solr的核心名称。默认情况下，它是collection1。

nutch索引因io异常而失败

1 个答案: