Question

我正在研究AWS Java S3库。

这是我的代码，它使用AWS的高级API将文件上传到s3。

        ClientConfiguration configuration = new ClientConfiguration();
        configuration.setUseGzip(true);
        configuration.setConnectionTTL(1000 * 60 * 60);
        AmazonS3Client amazonS3Client = new AmazonS3Client(configuration);
        TransferManager transferManager = new TransferManager(amazonS3Client);

        ObjectMetadata objectMetadata = new ObjectMetadata();
        objectMetadata.setContentLength(message.getBodyLength());
        objectMetadata.setContentType("image/jpg");

        transferManager.getConfiguration().setMultipartUploadThreshold(1024 * 10);

        PutObjectRequest request = new PutObjectRequest("test", "/image/test", inputStream, objectMetadata);
        request.getRequestClientOptions().setReadLimit(1024 * 10);
        request.setSdkClientExecutionTimeout(1000 * 60 * 60);

        Upload upload = transferManager.upload(request);
        upload.waitForCompletion();

我正在尝试上传大文件。它工作正常，但有时我遇到错误。我已将readLimit设置为（1024 * 10）。

2019-04-05 06:41:05,679 ERROR [com.demo.AwsS3TransferThread] (Aws-S3-upload) Error in saving File[media/image/osc/54/54ec3f2f-a938-473c-94b7-a55f39aac4a6.png] on S3[demo-test]: com.amazonaws.ResetException: Failed to reset the request input stream;  If the request involves an input stream, the maximum stream buffer size can be configured via request.getRequestClientOptions().setReadLimit(int)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.resetRequestInputStream(AmazonHttpClient.java:1221)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1042)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:948)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:661)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:635)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:618)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$300(AmazonHttpClient.java:586)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:573)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:445)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4041)
    at com.amazonaws.services.s3.AmazonS3Client.doUploadPart(AmazonS3Client.java:3041)
    at com.amazonaws.services.s3.AmazonS3Client.uploadPart(AmazonS3Client.java:3026)
    at com.amazonaws.services.s3.transfer.internal.UploadCallable.uploadPartsInSeries(UploadCallable.java:255)
    at com.amazonaws.services.s3.transfer.internal.UploadCallable.uploadInParts(UploadCallable.java:189)
    at com.amazonaws.services.s3.transfer.internal.UploadCallable.call(UploadCallable.java:121)
    at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:139)
    at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:47)

readLimit的用途是什么？它将如何有用？我应该怎么做才能避免这种异常？

Answer 1

对此进行了1周的研究后，我发现，如果您上传的文件大小小于48GB，则可以将readLimit值设置为5.01MB。

因为AWS将文件分为多个部分，每个部分的大小值为5MB（如果您未更改最小部分大小值）。根据AWS规范，最后部件的大小小于5MB。因此我将readLimit设置为5MB即可解决此问题。

InputStream readLimit用途：

标记此输入流中的当前位置。随后对reset方法的调用将流重新定位在最后标记的位置，以便后续读取重新读取相同的字节。Readlimit参数告诉此输入流允许在标记位置无效之前读取许多字节。 mark的一般约定是，如果方法markSupported返回，则流将以某种方式记住在调用mark之后读取的所有字节，并准备在每次调用方法reset时再次提供相同的字节。但是，如果在调用复位之前从流中读取了readLimit个字节以上，则流根本不需要记住任何数据。

为什么我们需要在AWS S3 Java客户端中设置setReadLimit（int）

1 个答案: