java.util.zip.ZipEntry和apache压缩ZipEntry损坏了一些图像文件

Question

我一直在使用Spring的多部分上传器控制器来上传和存储压缩文件中的条目，但我发现occaisional PNG文件已被破坏，而不是在其byte []中使用类似“PNG ...”的内容，它以“fþ»ÀÃgÞÉ”或类似的方式开头。这似乎发生在每次运行的相同文件中。我使用java.util.ZipEntry尝试了所有这些，然后我尝试了Apache Compress，发现Apache将损坏的不同文件压缩到Java 7实用程序，但在后续运行中始终使用相同的文件。

代码（首先是java.util.zip.ZipEntry）：

protected void processZipFile(String path, MultipartFile file, String signature) throws IOException {

    DateFormat df = new SimpleDateFormat("yyyyMMddhhmmss");
    File tempFile = new File(System.getProperty("user.dir") + "/" + file.getName() + df.format(new Date()));
    file.transferTo(tempFile);

    ZipFile zipFile = null;
    try {
        zipFile = new ZipFile(tempFile);

        LOG.debug("Processing archive with name={}, size={}.", file.getName(), file.getSize());

        final Enumeration<? extends ZipEntry> entries = zipFile.entries();
        while ( entries.hasMoreElements() )
        {
            ZipEntry entry = entries.nextElement();

            LOG.debug("Processing file={} is directory?={}.", entry.getName(), entry.isDirectory());

            // we don't bother processing directories, and we don't process any resource fork info
            // from Mac OS X (which does not seem to be transparent to ZipFile).

            if (!(entry.isDirectory() || entry.getName().contains("__MACOSX")  || entry.getName().contains(".DS_Store"))) {
                // if the entry is a file, extract it


                Content contentToSave = null;

                if(entry.getName().contains("gif") || entry.getName().contains("png") || entry.getName().contains("jpeg")) {

                    byte[] bytes = readInputStream( zipFile.getInputStream( entry ), entry.getSize() );

                    LOG.debug("{} is of inflated-length={} from compressed-length={}",
                            entry.getName(), bytes.length, entry.getCompressedSize());

                    if(entry.getName().contains("gif")) {
                        contentToSave = Content.makeImage(path + entry.getName(), Content.GIF, signature, bytes);

                    } else if (entry.getName().contains("png")) {
                        contentToSave = Content.makeImage(path + entry.getName(), Content.PNG, signature, bytes);

                    } else if (entry.getName().contains("jpeg")) {
                        contentToSave = Content.makeImage(path + entry.getName(), Content.JPEG, signature, bytes);

                    }
                } else {

                    InputStream is = zipFile.getInputStream(entry);

                    if (entry.getName().contains("json")) {
                        contentToSave = Content.makeFile(path + entry.getName(), Content.JSON, signature, convertStreamToString(is));
                    } else if (entry.getName().contains("js")) {
                        contentToSave = Content.makeFile(path + entry.getName(), Content.JS, signature, convertStreamToString(is));
                    } else if (entry.getName().contains("css")) {
                        contentToSave = Content.makeFile(path + entry.getName(), Content.CSS, signature, convertStreamToString(is));
                    } else if (entry.getName().contains("xml")) {
                        contentToSave = Content.makeFile(path + entry.getName(), Content.XML, signature, convertStreamToString(is));
                    } else if (entry.getName().contains("html")) {
                        contentToSave = Content.makeFile(path + entry.getName(), Content.HTML, signature, convertStreamToString(is));
                    }
                }

                contentService.putOrReplace(contentToSave);
                LOG.info("Persisted file: {} from uploaded version.", contentToSave.getName());
            }

        }
    } catch (ZipException e) {
        // If I can't create a ZipFile, then this is not a zip file at all and it cannot be processed
        // by this method. Its pretty dumb that there's no way to determine whether the contents are zipped through
        // the ZipFile API, but that's just one of its many problems.
        e.printStackTrace();
        LOG.error("{} is not a zipped file, or it is empty", file.getName());
    } finally {
        zipFile = null;
    }
    tempFile.delete();

}

现在org.apache.commons.compress.archivers.zip.ZipFile也是如此：

protected void processZipFile(String path, MultipartFile file, String signature) throws IOException {

    DateFormat df = new SimpleDateFormat("yyyyMMddhhmmss");
    File tempFile = new File(System.getProperty("user.dir") + "/" + file.getName() + df.format(new Date()));
    file.transferTo(tempFile);

    ZipFile zipFile = null;
    try {
        zipFile = new ZipFile(tempFile);

        LOG.debug("Processing archive with name={}, size={}.", file.getName(), file.getSize());

        final Enumeration<? extends ZipArchiveEntry> entries = zipFile.getEntries();
        while ( entries.hasMoreElements() ) {
            ZipArchiveEntry entry = entries.nextElement();

            LOG.debug("Processing file={} is directory?={}.", entry.getName(), entry.isDirectory());

                // we don't bother processing directories, and we don't process any resource fork info
                // from Mac OS X (which does not seem to be transparent to ZipFile).

                if (!(entry.isDirectory() || entry.getName().contains("__MACOSX")  || entry.getName().contains(".DS_Store"))) {
                    // if the entry is a file, extract it

                    Content contentToSave = null;

                    if(entry.getName().contains("gif") || entry.getName().contains("png") || entry.getName().contains("jpeg")) {

                        byte[] bytes = readInputStream( zipFile.getInputStream( entry ), entry.getSize() );

                        LOG.debug("{} is of inflated-length={} from compressed-length={}",
                            entry.getName(), bytes.length, entry.getCompressedSize());

                        if(entry.getName().contains("gif")) {
                            contentToSave = Content.makeImage(path + entry.getName(), Content.GIF, signature, bytes);

                        } else if (entry.getName().contains("png")) {
                            contentToSave = Content.makeImage(path + entry.getName(), Content.PNG, signature, bytes);

                        } else if (entry.getName().contains("jpeg")) {
                            contentToSave = Content.makeImage(path + entry.getName(), Content.JPEG, signature, bytes);

                        }
                    } else {

                        InputStream is = zipFile.getInputStream(entry);

                        if (entry.getName().contains("json")) {
                            contentToSave = Content.makeFile(path + entry.getName(), Content.JSON, signature, convertStreamToString(is));
                        } else if (entry.getName().contains("js")) {
                            contentToSave = Content.makeFile(path + entry.getName(), Content.JS, signature, convertStreamToString(is));
                        } else if (entry.getName().contains("css")) {
                            contentToSave = Content.makeFile(path + entry.getName(), Content.CSS, signature, convertStreamToString(is));
                        } else if (entry.getName().contains("xml")) {
                            contentToSave = Content.makeFile(path + entry.getName(), Content.XML, signature, convertStreamToString(is));
                        } else if (entry.getName().contains("html")) {
                            contentToSave = Content.makeFile(path + entry.getName(), Content.HTML, signature, convertStreamToString(is));
                        }
                    }

                    contentService.putOrReplace(contentToSave);
                    LOG.info("Persisted file: {} from uploaded version.", contentToSave.getName());
                }

        }
    } catch (ZipException e) {
        e.printStackTrace();
        LOG.error("{} is not a zipped file, or it is empty", file.getName());
    } catch (IOException e) {
        e.printStackTrace();
        LOG.error("{} is not a file, or it is empty", file.getName());
    } finally {
        zipFile = null;
    }
    tempFile.delete();
}

这两种被称为方法的方法是：

private static byte[] readInputStream( final InputStream is, final long length ) throws IOException {
    final byte[] buf = new byte[ (int) length ];
    int read = 0;
    int cntRead;
    while ( ( cntRead = is.read( buf, 0, buf.length ) ) >=0  )
    {
        read += cntRead;
    }
    return buf;
}

和

public String convertStreamToString(InputStream is) throws IOException {
    StringBuilder sb = new StringBuilder(2048);
    char[] read = new char[128];
    try (InputStreamReader ir = new InputStreamReader(is, StandardCharsets.UTF_8)) {
        for (int i; -1 != (i = ir.read(read)); sb.append(read, 0, i));
    }

    // need to remove the ? at teh beginning of some files. This comes from the UTF8 BOM
    // that is added to some files saved as UTF8
    String out = sb.toString();
    String utf8Bom = new String(new char[]{'\ufeff'});
    if(out.contains(utf8Bom)) {
        out = out.replace(utf8Bom,"");
    }
    return out;
}

第二个当然不太可能是问题的一部分。

我已经搜索过了，看起来已经发现了与此类似的问题，但它一直是一些外部问题。有谁知道为什么会这样？

我重新编辑了一些图像，发现如果我将图像更改为黑白，或者更改整个图像的色调，问题就会消失，但如果我添加边框或更改单一颜色问题遗迹。看起来某些文件中特定的字节排列会勾勒出Java自己和Apache压缩文件读取器使用的任何底层API中的错误，但这只是推测。

编辑：其他用法表明损坏发生在大小超过10K的gif中，所以这可能与bug有关吗？我试图在ReadInputStream（）调用中任意加倍缓冲区的大小，但除了在特殊大图像中溢出MySQL中的blob大小（49K变为98K，这太​​大了）之外什么也没做。

com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'encoded_content' at row 1

Answer 1

我的发现是，这个问题出现在“打包大小”的情况下。比实际大小大，这可能发生在png文件中，例如已经压缩过的＆＃39;他们自己。