Question

我已经使用名称将文件保存在AWS S3上，以模拟文件夹结构。

例如sample.jpeg文件夹下的文件ABC将被命名为ABC/sample.jpeg

我正在尝试使用Java SDK v2以编程方式删除这些文件夹。我正在尝试使用prefix列出这些文件。

例如在上面的示例中，我尝试列出存储桶中所有带有前缀ABC的文件。

有问题的是，如果没有此前缀，SDK会列出文件，但是如下面的代码所示，我的ListObjectsRequest中的前缀会抛出s3Exception。异常本身具有awsErrorDetails属性，该属性具有sdkHttpResponse包装statusCode：302和statusText："Moved Temporarily"

以下是异常跟踪：

software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 302, Request ID: null)
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.handleErrorResponse(HandleResponseStage.java:115) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.handleResponse(HandleResponseStage.java:73) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:58) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:41) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:64) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:36) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.doExecute(RetryableStage.java:113) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.execute(RetryableStage.java:86) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:62) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:42) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:57) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:37) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:240) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.client.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:96) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.client.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:120) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.client.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:73) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:44) ~[sdk-core-2.7.22.jar:na]
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55) ~[aws-core-2.7.22.jar:na]
at software.amazon.awssdk.services.s3.DefaultS3Client.listObjects(DefaultS3Client.java:2278) ~[s3-2.7.22.jar:na]

这是我正在使用的代码段：

    ListObjectsRequest listObjectsRequest = ListObjectsRequest.builder()
            .bucket(this.bucketName)
            .prefix(path)
            .build();

    List<ObjectIdentifier> objectIds = s3client.listObjects(listObjectsRequest)
            .contents()
            .stream()
            .map(s3Object -> ObjectIdentifier.builder().key(s3Object.key()).build())
            .collect(toList());

以下是我的pom.xml的依赖项：

      <properties>
           <aws-sdk.version>2.7.22</aws-sdk.version>
      </properties>

    ...

        <!-- Amazon Web Services -->
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>s3</artifactId>
            <version>${aws-sdk.version}</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>auth</artifactId>
            <version>${aws-sdk.version}</version>
        </dependency>

Answer 1

看到我们在评论中的讨论没有进行到任何地方，我宁愿看到这种赏金被浪费掉。

我已经在您的计算机上运行了您的代码（使用相同的AWS开发工具包版本），但无法重现该异常。如果您仍然遇到问题，建议您在GitHub存储库上打开一个问题来解释您的问题：https://github.com/aws/aws-sdk-java-v2/issues/new

我以前已经经历过这个过程，他们通常会在几天内做出答复。

如果他们设法解决您的问题，请报告！

Answer 2

问题在于aws对org.apache.httpcomponents:httpclient的内部依赖性。

httpclient的最低要求版本为4.5.9。如果您在项目pom中降级该版本，则将出现此问题。

要检查您正在运行的版本，请尝试：

mvn dependency:tree

找出httpclient版本。这是我的样子：

[INFO] |  +- software.amazon.awssdk:apache-client:jar:2.8.4:runtime
[INFO] |  |  +- org.apache.httpcomponents:httpclient:jar:4.5.10:runtime
[INFO] |  |  |  \- commons-codec:commons-codec:jar:1.11:runtime
[INFO] |  |  \- org.apache.httpcomponents:httpcore:jar:4.4.12:runtime

解决方案：
如果您已经通过使用任何Bom或依赖项管理降级了版本（在我的情况下，spring-boot-starter-parent:2.1.5.RELEASE导致这种情况发生），则只需将其添加到pom中就可以了：

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

相关问题：
Issue#1434，Issue#1919，Issue#1372

Answer 3

是否可以在s3client的实例之间进行比较（在旧版本和有问题的新版本Java的AWS开发工具包之间）。请分享。

可能必须调试和检查。这两个值之间的默认值可能会有所不同。例如，如果默认区域不同，则将不会使用新的SDK获得以前保存的对象，而无需进行任何更改。

此外，由于您要迁移到主要版本，因此很可能会发现更多问题。它可以指导您参考Migration Guide。

如果您只想查找特定更改，则可以选中此What's Different Page。请检查“更改日志”页面上的链接，您将获得按方法比较的更改。

Answer 4

我之前确实遇到过类似的错误。造成该错误的原因是，AWS员工已开始将其API转换为2.x SDK的v2。下面是无论是否带前缀都可以完美使用的代码段。

ListObjectsV2Request request = new ListObjectsV2Request().withBucketName(bucketName).withPrefix(prefix)
                    .withDelimiter(GenericConstants.FORWARD_SLASH);
ListObjectsV2Result result = s3Client.listObjectsV2(request);

如有需要，随时询问进一步的细节。我对S3的v2 API做了很多工作。

Amazon SDK Java v2 S3Client listObjects引发S3Exception

4 个答案: