Question

我正在使用Java Amazon SDK与S3一起存储上传的文件。我想保留原始文件名，我将它放在密钥的末尾，但我也使用虚拟目录结构 - 类似于<dirname>/<uuid>/<originalFilename>。

问题在于，当我想使用api生成预先下载的URL以供下载时使用：

URL url = s3Client.generatePresignedUrl(generatePresignedUrlRequest);
return url.toExternalForm();

sdk url转义整个键，包括斜杠。虽然它仍然有效，但这意味着下载的文件名包含整个密钥，而不仅仅是最后的原始文件名位。我知道应该可以在不转义斜杠的情况下执行此操作，但我正在尝试避免重写SDK中已有的大量代码。这有一个共同的解决方案吗？我知道我使用的网络应用程序遵循相同的模式，并且没有斜线转义问题。

Answer 1

这是当前Java SDK中的一个错误：

如果你看https://github.com/aws/aws-sdk-java/blob/master/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L2820

内部调用的方法presignRequest具有以下代码：

    String resourcePath = "/" +
        ((bucketName != null) ? bucketName + "/" : "") +
        ((key != null) ? ServiceUtils.urlEncode(key) : "") +
        ((subResource != null) ? "?" + subResource : "");

在签名之前，密钥是在此编码的URL，我认为这是错误。

您可以从AmazonS3Client继承并覆盖功能以解决此问题。

在某些地方，建议使用url.getQuery()并将其作为原始awsURL（https://forums.aws.amazon.com/thread.jspa?messageID=356271）的前缀。但是，正如您所说，这会产生错误，因为资源键与签名不匹配。

以下问题可能也有关系，我没有查看建议的workarround：

How to generate pre-signed Amazon S3 url for a vanity domain, using amazon sdk?

亚马逊在之前认识并修复了类似的错误： https://forums.aws.amazon.com/thread.jspa?messageID=418537

所以我希望它会在下一个版本中修复。

Answer 2

我仍然希望有一个比这更好的解决方案，但是看到@aKzenT已经证实我的结论是没有现成的解决方案，我写了一个。它只是AmazonS3Client的一个简单子类。我担心它很脆弱，因为我必须从我覆盖的方法中复制大量代码，但它似乎是最小的解决方案。我可以确认它在我自己的代码库中工作正常。我在gist中发布了代码，但为了得到完整答案：

import com.amazonaws.AmazonWebServiceRequest;
import com.amazonaws.HttpMethod;
import com.amazonaws.Request;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.handlers.RequestHandler;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.Headers;
import com.amazonaws.services.s3.internal.S3QueryStringSigner;
import com.amazonaws.services.s3.internal.ServiceUtils;

import java.util.Date;

/**
 * This class should be a drop in replacement for AmazonS3Client as long as you use the single credential
 * constructor. It could probably be modified to add additional constructors if needed, but this is the one we use.
 * Supporting all of them didn't seem trivial because of some dependencies in the original presignRequest method.
 *
 * The only real purpose of this class is to change the behavior of generating presigned URLs. The original version
 * escaped slashes in the key and this one does not. Pretty url paths are kept intact.
 *
 * @author Russell Leggett
 */
public class PrettyUrlS3Client extends AmazonS3Client{
    private AWSCredentials awsCredentials;

    /**
     * This constructor is the only one provided because it is only one I needed, and it
     * retains awsCredentials which might be needed in the presignRequest method
     *
     * @param awsCredentials
     */
    public PrettyUrlS3Client(AWSCredentials awsCredentials) {
        super(awsCredentials);
        this.awsCredentials = awsCredentials;
    }

    /**
     * WARNING: This method is an override of the AmazonS3Client presignRequest
     * and copies most of the code. Should be careful of updates to the original.
     *
     * @param request
     * @param methodName
     * @param bucketName
     * @param key
     * @param expiration
     * @param subResource
     * @param <T>
     */
    @Override
    protected <T> void presignRequest(Request<T> request, HttpMethod methodName, String bucketName, String key, Date expiration, String subResource) {

        // Run any additional request handlers if present
        if (requestHandlers != null) {
            for (RequestHandler requestHandler : requestHandlers) {
                requestHandler.beforeRequest(request);
            }
        }
        String resourcePath = "/" +
                ((bucketName != null) ? bucketName + "/" : "") +
                ((key != null) ? keyToEscapedPath(key)/* CHANGED: this is the primary change */ : "") +
                ((subResource != null) ? "?" + subResource : "");

        //the request apparently needs the resource path without a starting '/'
        request.setResourcePath(resourcePath.substring(1));//CHANGED: needed to match the signature with the URL generated from the request
        AWSCredentials credentials = awsCredentials;
        AmazonWebServiceRequest originalRequest = request.getOriginalRequest();
        if (originalRequest != null && originalRequest.getRequestCredentials() != null) {
            credentials = originalRequest.getRequestCredentials();
        }

        new S3QueryStringSigner<T>(methodName.toString(), resourcePath, expiration).sign(request, credentials);

        // The Amazon S3 DevPay token header is a special exception and can be safely moved
        // from the request's headers into the query string to ensure that it travels along
        // with the pre-signed URL when it's sent back to Amazon S3.
        if (request.getHeaders().containsKey(Headers.SECURITY_TOKEN)) {
            String value = request.getHeaders().get(Headers.SECURITY_TOKEN);
            request.addParameter(Headers.SECURITY_TOKEN, value);
            request.getHeaders().remove(Headers.SECURITY_TOKEN);
        }
    }

    /**
     * A simple utility method which url escapes an S3 key, but leaves the
     * slashes (/) unescaped so they can stay part of the url.
     * @param key
     * @return
     */
    public static String keyToEscapedPath(String key){
        String[] keyParts = key.split("/");
        StringBuilder result = new StringBuilder();
        for(String part : keyParts){
            if(result.length()>0){
                result.append("/");
            }
            result.append(ServiceUtils.urlEncode(part));
        }
        return result.toString().replaceAll("%7E","~");
    }
}

更新我更新了要点和此代码，以解决我遇到的问题。即使使用标准客户端也会发生这种情况，但无法解决问题。有关详细信息，请参阅要点/跟踪我可能做出的任何进一步更改。

Answer 3

Java SDK的1.4.3版本似乎已经解决了这个问题。也许，它已经被修复了，但我可以确认它在1.4.3中正常工作。

Amazon S3 Presigned URL从密钥中删除斜杠

3 个答案: