Question

我在Java中有一个非常简单的代码，它从hdfs

读取数据

try{
    InputStream s = new GzipCompressorInputStream(hdfsFileSystem.open(filePath), false);
    ByteStreams.copy(s, outputStream);
    s.close();
}
catch (Exception ex){
    logger.error("Problem with file "+ filePath,ex);
}

有时（并非总是）它会抛出异常

java.lang.NoSuchMethodError: org.apache.commons.io.IOUtils.closeQuietly(Ljava/io/Closeable;)V
        at org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:1099)
        at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:533)
        at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:749)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:793)
        at java.io.DataInputStream.read(DataInputStream.java:149)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.init(GzipCompressorInputStream.java:136)
        at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.<init>(GzipCompressorInputStream.java:129)
        [...]

在线下方：

InputStream s = new GzipCompressorInputStream(hdfsFileSystem.open(filePath), false);

我正在使用bellow maven依赖来加载hadoop客户端：

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>2.2.0</version>
    </dependency>

有人知道如何解决这个问题吗？我可以将catch(Exception e)更改为catch(Error e)，但这不是解决方法。

Answer 1

在你的课堂路径中看起来有几个＆＃34; commons-io.jar＆＃34;有不同的版本。方法＆＃34; closeQuietly（Ljava / io / Closeable;）＆＃34;出现在2.0版本中。有时＆＃34; commons-io.jar＆＃34;首先加载旧版本，并出现异常。需要修复类路径。

org.apache.hadoop.hdfs.DFSInputStream中的NoSuchMethodError

1 个答案: