我是Apache nutch的新手,所以我遵循Apache wiki的Apache nutch教程
环境是:
Windows 10
Jdk 1.8
Cygin 2.9.0
Apache Nutch 1.14
Solr 6.6.2
winutils - hadoop-2.8.1
我已经设置了JAVA_HOME和HADOOP_HOME
当我在Cygin中运行crawl命令时
./crawl -s urls mycrawdir 3
出现以下错误
注入种子URL
/home/apache-nutch-1.14/bin/nutch注入mycrawdir / crawldb网址
喷油器:从2018-01-19 20:55:48开始
注入器:crawlDb:mycrawdir / crawldb
注射器:urlDir:urls
Injector:将注入的url转换为抓取db条目。
Exception in thread "main" java.lang.UnsatisfiedLinkError: <br>org.apache.hadoop.io. nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:6 09)
at org.apache.hadoop.fs.FileUtil.canRead(FileUtil.java:977)
at org.apache.hadoop.util.DiskChecker.checkAccessByFileMethods(DiskCheck er.java:187)
at org.apache.hadoop.util.DiskChecker.checkDirAccess(DiskChecker.java:17 4)
at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:108)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChange d(LocalDirAllocator.java:285)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPa thForWrite(LocalDirAllocator.java:344)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirA llocator.java:150)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirA llocator.java:131)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirA llocator.java:115)
at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup(LocalDist ributedCacheManager.java:125)
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.jav a:163)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java :731)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitt er.java:240)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma tion.java:1746)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
at org.apache.nutch.crawl.Injector.inject(Injector.java:417)
at org.apache.nutch.crawl.Injector.run(Injector.java:563)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.crawl.Injector.main(Injector.java:528)
Error running:
/home/apache-nutch-1.14/bin/nutch inject mycrawdir/crawldb urls
Failed with exit value 1.