Question

这个问题是我之前在Java OutOfMemoryError while merge large file parts from chunked files

的帖子中的续集问题

我正在成功实现答案中的代码，并将大块文件合并到单个文件中，而不会出现OOME错误。 但是，该文件总是损坏并且与原始文件不同。

这是我之前提问的原始代码

 private void mergeFile(String identifier, int totalFile, String outputFile) throws AppException{
    File[] fileDatas = new File[totalFile];
    byte fileContents[] = null;
    int totalFileSize = 0;
    int filePartUploadSize = 0;
    int tempFileSize = 0;

    for (int i = 0; i < totalFile; i++) {
        fileDatas[i] = new File(identifier + "." + (i + 1));
        totalFileSize += fileDatas[i].length();
    }

    try {
        fileContents = new byte[totalFileSize]; // allocating memory to contain all the byte stream for further writing
        InputStream inStream;
        for (int j = 0; j < totalFile; j++) {
            inStream = new BufferedInputStream(new FileInputStream(fileDatas[j]));
            filePartUploadSize = (int) fileDatas[j].length();
            inStream.read(fileContents, tempFileSize, filePartUploadSize);
            tempFileSize += filePartUploadSize;
            inStream.close();
        }
    } catch (FileNotFoundException ex) {
        throw new AppException(AppExceptionCode.FILE_NOT_FOUND);
    } catch (IOException ex) {
        throw new AppException(AppExceptionCode.ERROR_ON_MERGE_FILE);
    } finally {
        write(fileContents, outputFile);
        for (int l = 0; l < totalFile; l++) {
            fileDatas[l].delete();
        }
    }
}

private  void write(byte[] DataByteArray, String DestinationFileName) throws AppException{
    try {
        OutputStream output = null;
        try {
            output = new BufferedOutputStream(new FileOutputStream(DestinationFileName));
            output.write(DataByteArray);
        } finally {
            output.close();
        }
    } catch (FileNotFoundException ex) {
        throw new AppException(AppExceptionCode.FILE_NOT_FOUND, ex);
    } catch (IOException ex) {
        throw new AppException(AppExceptionCode.ERROR_ON_WRITE_FILE, ex);
    }
}

正如您所看到的，从上一个问题开始，我正在分配大量的字节数据，以便包含所有文件，然后将其写入单个文件中。拥有这种技术将使我受益，避免在合并文件时避免损坏的数据。但是，这会在合并大文件（＆gt; 1 GB）时给我OOME（OutOfMemoryError）。

这是使用Jim回答的新合并方法

private void merge(String identifier, int totalFile, String outputFile) throws AppException{
    ArrayList<File> files = new ArrayList<>();// put your files here
    for (int i = 0; i < totalFile; i++) {
        files.add(new File(identifier + "." + (i + 1)));
    }

    File output = new File(outputFile);
    BufferedOutputStream boss = null;
    try {
        boss = new BufferedOutputStream(new FileOutputStream(output));
        for (File file : files) {
            BufferedInputStream bis = null;
            try {
                bis = new BufferedInputStream(new FileInputStream(file));
                boolean done = false;
                while (!done) {
                    int data = bis.read();
                    boss.write(data);
                    done = data < 0;
                }
            } catch (Exception e) {
                throw new AppException(AppExceptionCode.FILE_NOT_FOUND, e.getMessage());
            } finally {
                try {
                    bis.close();//do this in a try catch just in case
                } catch (Exception e) {
                    throw new AppException(AppExceptionCode.ERROR_ON_MERGE_FILE,e.getMessage());
                }
            }
        }
    } catch (Exception e) {
        throw new AppException(AppExceptionCode.ERROR_ON_MERGE_FILE,e.getMessage());
    } finally {
        try {
            boss.close();
        } catch (Exception e) {
            throw new AppException(AppExceptionCode.ERROR_ON_MERGE_FILE,e.getMessage());
        }
    }
    for (File file : files) {
        file.delete();
    }

}

此方法可以避免OOME异常，但文件总是损坏...

你能告诉我这个方法的根本错误，或者你可以给我一个更好的代码...... 非常感谢....

Answer 1

可能会像这样改变循环来解决问题。

        int data = -1;
        while ((data = bis.read()) !=-1) {
            boss.write(data);
        }

这样可以减少将无效字符写入文件的机会。

Java - 无法从分块文件合并文件

1 个答案: