使用预先签名的URL上传到带有curl的s3(获得403)

时间:2012-01-30 17:24:43

标签: java bash curl amazon-s3

我正在使用curl调用Java ReST API来检索URL。然后,Java使用我的S3凭据生成用于S3上载的预签名URL,并在ReST回复中返回该URL。 Curl获取URL并使用它上传到S3,但S3返回403“我们计算的请求签名与您提供的签名不匹配。请检查您的密钥和签名方法。”

以下是我用来生成预签名网址的代码:

public class S3Util {
    static final AmazonS3 s3 = new AmazonS3Client( new AWSCredentials() {
        @Override
        public String getAWSAccessKeyId() {
            return "XXXXXXX";
        }
        @Override
        public String getAWSSecretKey() {
            return "XXXXXXXXXXXXXX";
        }
    });
    static final String BUCKET = "XXXXXXXXXXXXXXXXXXXXXXXXXXX";

    static public URL getMediaChunkURL( MediaChunk mc, HttpMethod method ) {
        String key = ...
        //way in the future (for testing)...
        Date expiration = new Date( System.currentTimeMillis() + CalendarUtil.ONE_MINUTE_IN_MILLISECONDS*60*1000 );

        GeneratePresignedUrlRequest req = new GeneratePresignedUrlRequest(BUCKET, key, method);
        req.setExpiration(expiration);
        req.addRequestParameter("Content-Type", "application/octet-stream");

        //this gets passed to the end user:
        return s3.generatePresignedUrl(req);
    }
}

并且在curl中,从bash运行,我执行:

echo Will try to upload chunk to ${location}
curl -i -X POST \
        -F 'Content-Type=application/octet-stream' \
        -F "file=@${fileName}" \
        ${location} || (echo upload chunk failed. ; exit 1 )

除此之外,我尝试过PUT,我尝试过“Content-type”(小写T)。我意识到我错过了一些显而易见的东西(或者某些东西),但在阅读了相应的文档,谷歌搜索并查看了许多类似的问题后,我不确定那是什么。我看到很多关于必需标题的提示,但我认为重新签名的URL应该可以消除这些需求。也许不是吗?

TIA!

更新

为了清楚起见,我测试了下载量,并且运行正常。

Java看起来像:

GeneratePresignedUrlRequest req = new GeneratePresignedUrlRequest(BUCKET, key, HttpMethod.GET);
req.setExpiration(expiration);

和卷曲很简单:

curl -i ${location}

3 个答案:

答案 0 :(得分:24)

我已经能够通过C#生成预先签名的URL,然后按照预期通过curl上传。根据我的测试,我怀疑你确实没有正确使用 curl - 我已经能够像这样上传一个文件:

curl -v --upload-file ${fileName} ${location}

参数-v转储请求和响应头(以及SSL握手)以进行调试和说明:

> PUT [...] HTTP/1.1
> User-Agent: curl/7.21.0 [...]
> Host: [...]
> Accept: */*
> Content-Length: 12
> Expect: 100-continue

请注意,--upload-file(或-T)可以按预期方式PUT,但会根据需要添加更多标题,从而产生正确的回复:

< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< x-amz-id-2: [...]
< x-amz-request-id:  [...]
< Date: Tue, 31 Jan 2012 18:34:56 GMT
< ETag: "253801c0d260f076b0d5db5b62c54824"
< Content-Length: 0
< Server: AmazonS3

答案 1 :(得分:1)

生成网址的方式:

private static URL generateRUL(String objectKey, String ACCESS_KEY, String SECRET_KEY, String BUCKET_NAME) {
    AmazonS3 s3Client = new AmazonS3Client(new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY));
    URL url = null;

    try {
        GeneratePresignedUrlRequest request  = new GeneratePresignedUrlRequest(BUCKET_NAME, objectKey);
        request.setMethod(com.amazonaws.HttpMethod.PUT);
        request.setExpiration(new Date( System.currentTimeMillis() + (60 * 60 * 1000)));

        // Very important ! It won't work without adding this! 
        // And request.addRequestParameter("Content-Type", "application/octet-stream") won't work neither
        request.setContentType("application/octet-stream");

        url = s3Client.generatePresignedUrl(request ); 
    } catch (AmazonServiceException exception) { 
    } catch (AmazonClientException ace) { }

    return url;
}

上传文件的方式:

public int upload(byte[] fileBytes, URL url) {
    HttpURLConnection connection = (HttpURLConnection) url.openConnection();
    connection.setDoOutput(true);
    connection.setRequestMethod("PUT");
    connection.setRequestProperty("Content-Type", "application/octet-stream"); // Very important ! It won't work without adding this!
    OutputStream output = connection.getOutputStream();

    InputStream input = new ByteArrayInputStream(fileBytes);
    byte[] buffer = new byte[4096];
    int length;
    while ((length = input.read(buffer)) > 0) {
        output.write(buffer, 0, length);
    }
    output.flush();

    return connection.getResponseCode();
}

答案 2 :(得分:-1)

尽管GeneratePresignedUrlRequest接受了一个http方法参数(并且具有setMethod函数),但除了GET之外,它似乎无法使用。

http://wiki.nercomp.org/wiki/images/0/05/AmazonWebServices.pdf声明“签署请求并将其提供给第三方执行的做法仅适用于简单的对象GET请求。”也许设置另一种方法可以用于某些事情,但显然不是这样。

所以,相反,我必须按照这里的说明进行操作:

http://aws.amazon.com/articles/1434?_encoding=UTF8&jiveRedirect=1

这更复杂,因为客户需要发布完整的表单,而不仅仅是使用URL,还意味着所有帖子信息必须单独传达给客户端,但它似乎确实有效。 / p>