Question

当我使用Nutch 1.15开始抓取时，

/usr/local/nutch/bin/crawl --i -s urls/seed.txt crawldb 5

然后它开始运行，尝试获取时出现此错误：

2019-02-10 15:29:32,021 INFO  mapreduce.Job - Running job: job_local1267180618_0001
2019-02-10 15:29:32,145 INFO  fetcher.FetchItemQueues - Using queue mode : byHost
2019-02-10 15:29:32,145 INFO  fetcher.Fetcher - Fetcher: threads: 50
2019-02-10 15:29:32,145 INFO  fetcher.Fetcher - Fetcher: time-out divisor: 2
2019-02-10 15:29:32,149 INFO  fetcher.QueueFeeder - QueueFeeder finished: total 1 records hit by time limit : 0
2019-02-10 15:29:32,234 WARN  mapred.LocalJobRunner - job_local1267180618_0001
java.lang.Exception: java.lang.NullPointerException
        at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
        at org.apache.nutch.net.URLExemptionFilters.<init>(URLExemptionFilters.java:39)
        at org.apache.nutch.fetcher.FetcherThread.<init>(FetcherThread.java:154)
        at org.apache.nutch.fetcher.Fetcher$FetcherRun.run(Fetcher.java:222)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2019-02-10 15:29:33,023 INFO  mapreduce.Job - Job job_local1267180618_0001 running in uber mode : false
2019-02-10 15:29:33,025 INFO  mapreduce.Job -  map 0% reduce 0%
2019-02-10 15:29:33,028 INFO  mapreduce.Job - Job job_local1267180618_0001 failed with state FAILED due to: NA
2019-02-10 15:29:33,038 INFO  mapreduce.Job - Counters: 0
2019-02-10 15:29:33,039 ERROR fetcher.Fetcher - Fetcher job did not succeed, job status:FAILED, reason: NA
2019-02-10 15:29:33,039 ERROR fetcher.Fetcher - Fetcher: java.lang.RuntimeException: Fetcher job did not succeed, job status:FAILED, reason: NA
        at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:503)
        at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:543)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:517)

然后我在控制台中收到此错误，它是它运行的命令：

Error running:
  /usr/local/nutch/bin/nutch fetch -D mapreduce.job.reduces=2 -D mapred.child.java.opts=-Xmx1000m -D mapreduce.reduce.speculative=false -D mapreduce.map.speculative=false -D mapreduce.map.output.compress=true -D fetcher.timelimit.mins=180 crawlsites/segments/20190210152929 -noParsing -threads 50

Answer 1

我必须删除nutch文件夹并重新安装，然后才能工作。

如何修复运行Nutch 1.15错误fetcher.Fetcher的错误-提取器作业未成功，作业状态：失败，原因：不适用

1 个答案: