Question

我们正在尝试上传多个文件，即100个或更多文件到谷歌云存储，但我们得到了随机误差低于 平均文件大小：30MB - 40 MB

 08:57:15.488 [ForkJoinPool-1-worker-26] INFO  
 demo.storage.GCPStorageComponent - Error writing request body to
 server com.google.cloud.storage.StorageException: Error writing
 request body to server
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:189)
     at com.google.cloud.storage.spi.v1.HttpStorageRpc.write(HttpStorageRpc.java:581)
     at com.google.cloud.storage.BlobWriteChannel$1.run(BlobWriteChannel.java:51)
     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
     at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
at com.google.cloud.storage.BlobWriteChannel.flushBuffer(BlobWriteChannel.java:47)
     at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:160)
     at demo.storage.GCPStorageComponent.uploadFile(GCPStorageComponent.java:56)
     at demo.storage.StorageRepository.uploadFile(StorageRepository.java:56)
     at demo.restful.UploadRepository.uploadFileToBucket(UploadRepository.java:132)
     at demo.restful.UploadRepository.processClipResults(UploadRepository.java:56)
     at demo.restful.UploadRepository.lambda$getClipModelCallableList$0(UploadRepository.java:85)
     at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1424)
     at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
     at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
     at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
     at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
 Caused by: java.io.IOException: Error writing request body to server
     at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:3536)
     at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3519)
     at com.google.api.client.util.ByteStreams.copy(ByteStreams.java:55)
     at com.google.api.client.util.IOUtils.copy(IOUtils.java:94)
     at com.google.api.client.http.AbstractInputStreamContent.writeTo(AbstractInputStreamContent.java:72)
     at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:80)
     at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
     at com.google.cloud.storage.spi.v1.HttpStorageRpc.write(HttpStorageRpc.java:563)
     ... 17 more

以下是我们用于上传文件的代码。 我们使用多部分方法以1 MB的块上传文件。

public void uploadFile(String bucketName, String bucketFileName, Path localFilePath)
        throws IOException, StorageException {
    BlobInfo blobInfo = BlobInfo.newBuilder(bucketName, bucketFileName).build();
    ByteBuffer buffer = ByteBuffer.allocate(10_000_000); //i.e. 1 MB
    try (WriteChannel writer = storage.writer(blobInfo);
            ByteChannel inChannel = Files.newByteChannel(localFilePath);) { // write to bucket
        while (inChannel.read(buffer) > 0) {
            buffer.flip();
            writer.write(buffer);
            buffer.clear();
        }
    } catch (com.google.cloud.storage.StorageException ex) {
        LOGGER.info(ex.getMessage());
        ex.printStackTrace();
    }
}

修改：正在Google Cloud中进行访问已为同一

打开Github Issue

Answer 1

已通过解决此问题的错误的方式解决了此问题，该问题已合并到google cloud java的最新版本中。

这是提交链接https://github.com/GoogleCloudPlatform/google-cloud-java/pull/3433

链接以更改源代码https://github.com/GoogleCloudPlatform/google-cloud-java/pull/3433/commits/8e20b1c41e1ae99c4761f716c1ab4dc0f9fbf5e0

Google Cloud Storage同时为多个大型文件上传引发错误

1 个答案: