Question

我在电脑上运行Cloudera VM。我试图运行一个简单的map-reduce程序，它将从s3获得输入，并且我得到以下异常：

Exception in thread "main" java.lang.NoClassDefFoundError: org/jets3t/service/S3ServiceException at org.apache.hadoop.fs.s3native.NativeS3FileSystem.createDefaultStore(NativeS3FileSystem.java:242)
at org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:232)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:518)
at PopularHadoopMain.run(PopularHadoopMain.java:30)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at PopularHadoopMain.main(PopularHadoopMain.java:14)
Caused by: java.lang.ClassNotFoundException: org.jets3t.service.S3ServiceException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 13 more

我在安装了VM的eclipse上运行它我还添加了一行（在导入hadoop核心jar之后）：

job.getConfiguration().set("fs.s3n.impl", org.apache.hadoop.fs.s3native.NativeS3FileSystem.class.getName());

我已经阅读了这个内容，看起来我需要设置HADOOP_CLASSPATH，但我尝试了它并没有帮助。（我可能有那个故事）有人有想法吗？

Answer 1

错过jets3t

中的java.net jar

来自S3错误org / jets / service / S3ServiceException的Hadoop输入

1 个答案: