Appengine上的FileBackedOutputStream

Question

我在Appengine上的应用程序创建了一个包含更多65535行的csv文件

但是，编写时出现类型为OutOfMemoryError的错误：

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2271)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)

白色代码：

public static byte[] joinLines(Collection<String> lines) {
    final ByteArrayOutputStream stream = new ByteArrayOutputStream();

    boolean firstElement = true;

    for (final String part : lines) {
        String value = part + LINE_SEPARATOR;
        if (firstElement) {
            value = addExcelPrefix(value);
            firstElement = false;
        }

        final int currentSize = value.length();
        try {
            stream.write(value.getBytes(ENCODING), 0, currentSize); // OutOfMemoryError HERE
        } catch (UnsupportedEncodingException e) {
            LOGGER.info(e.getMessage());
        }
    }
    return stream.toByteArray();
}

所以我使用了Guava的FileBackedOutputStream来解决OutOfMemoryError的问题：

public static byte[] joinLines(Collection<String> lines) throws IOException {
    final FileBackedOutputStream stream = new FileBackedOutputStream(THRESHOLD, true);

    boolean firstElement = true;

    for (final String part : lines) {
        String value = part + LINE_SEPARATOR;
        if (firstElement) {
            value = addExcelPrefix(value);
            firstElement = false;
        }

        final int currentSize = value.length();
        try {
            stream.write(value.getBytes(ENCODING), 0, currentSize);
        } catch (IOException e) {
            LOGGER.error(e.getMessage());
        }
    }

    return stream.asByteSource().read();
}

但是，在appengine上，我现在在创建临时文件时出现类型为SecurityException的错误：

java.lang.SecurityException: Unable to create temporary file
    at java.io.File.checkAndCreate(File.java:2083)
    at java.io.File.createTempFile(File.java:2198)
    at java.io.File.createTempFile(File.java:2244)
    at com.google.common.io.FileBackedOutputStream.update(FileBackedOutputStream.java:196)
    at com.google.common.io.FileBackedOutputStream.write(FileBackedOutputStream.java:178)

如何允许使用FileBackedOutputStream在Appengine上创建临时文件？
在一个桶里，怎么样？

由于

Answer 1

我使用了解决我问题的GcsService：

protected String uploadBytesForCsv(Map<Integer, Map<Integer, Object>> rows) throws IOException {
    LOGGER.info("Get Bytes For Csv");

    final Collection<String> lines = cellsToCsv(rows);
    LOGGER.info("number line : " + lines.size());

    boolean firstElement = true;

    final String fileName = getFileName();

    final GcsFilename gcsFilename = new GcsFilename(config.getBucketName(), fileName);
    final GcsService gcsService = GcsServiceFactory.createGcsService();
    final GcsOutputChannel outputChannel = gcsService.createOrReplace(gcsFilename, GcsFileOptions.getDefaultInstance());

    for (final String part : lines) {
        final ByteArrayOutputStream stream = new ByteArrayOutputStream();
        String value = part + LINE_SEPARATOR;
        if (firstElement) {
            value = addExcelPrefix(value);
            firstElement = false;
        }

        final int currentSize = value.length();
        try {
            stream.write(value.getBytes(ENCODING), 0, currentSize);
            outputChannel.write(ByteBuffer.wrap(stream.toByteArray()));
        } catch (UnsupportedEncodingException e) {
            LOGGER.info(e.getMessage());
        }

        stream.flush();
        stream.close();
    }

    outputChannel.close();

    return new UrlBuilder(config.getStorageUrlForExport())
            .setBucketName(config.getBucketName())
            .setFilename(fileName).build();
}