Question

我正在创建一个只会将http请求转发到Amazon S3的代理。代理现在使用Nginx和ngx_aws_auth模块，该模块将必要的标头添加到HTTP请求中，因此客户端不需要。

Nginx配置：

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    client_body_in_file_only clean;
    client_body_buffer_size 32K;

    client_max_body_size 300M;

    send_timeout 300s;

    keepalive_timeout  65;

    gzip  on;

 server {
    listen     8000;

    location / {
      proxy_pass https://test-bucket.s3.amazonaws.com;

      aws_access_key xxxxx;
      aws_secret_key yyyyyy;
      s3_bucket test-bucket;

      proxy_set_header Authorization $s3_auth_token;
      proxy_set_header x-amz-date $aws_date;
    }
  }
}

除非我尝试上传更大的文件，否则几乎一切都很好。

GET：

$ curl -vX GET http://192.168.99.131:8000/a.txt ; echo
* About to connect() to 192.168.99.131 port 8000 (#0)
*   Trying 192.168.99.131...
* Connected to 192.168.99.131 (192.168.99.131) port 8000 (#0)
> GET /a.txt HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.99.131:8000
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.6.3
< Date: Sun, 30 Aug 2015 18:18:50 GMT
< Content-Type: application/x-www-form-urlencoded
< Content-Length: 1
< Connection: keep-alive
< x-amz-id-2: 1F/eQtkJU=
< x-amz-request-id: A970D9BCF5D5890B
< Last-Modified: Sat, 29 Aug 2015 00:58:33 GMT
< ETag: "0cc175b9c0f1b6a831c399e269772661"
< Accept-Ranges: bytes
<
* Connection #0 to host 192.168.99.131 left intact
a

PUT小文件：

$ echo 'a'>a;curl -vX PUT -d @a http://192.168.99.131:8000/a.txt
* About to connect() to 192.168.99.131 port 8000 (#0)
*   Trying 192.168.99.131...
* Connected to 192.168.99.131 (192.168.99.131) port 8000 (#0)
> PUT /a.txt HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.99.131:8000
> Accept: */*
> Content-Length: 1
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 1 out of 1 bytes
< HTTP/1.1 200 OK
< Server: nginx/1.6.3
< Date: Sun, 30 Aug 2015 18:20:03 GMT
< Content-Length: 0
< Connection: keep-alive
< x-amz-id-2: GF/hVpsr+r2R/j7qqUf0=
< x-amz-request-id: F499D02F2D28B6EB
< ETag: "0cc175b9c0f1b6a831c399e269772661"
<
* Connection #0 to host 192.168.99.131 left intact

在不调整标题的情况下输出较大的文件：

curl -vvvX PUT -d @1408133479717.jpg http://192.168.99.131:8000/a.jpg
* About to connect() to 192.168.99.131 port 8000 (#0)
*   Trying 192.168.99.131...
* Connected to 192.168.99.131 (192.168.99.131) port 8000 (#0)
> PUT /a.jpg HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.99.131:8000
> Accept: */*
> Content-Length: 320356
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Server: nginx/1.6.3
< Date: Sun, 30 Aug 2015 18:33:45 GMT
< Content-Length: 0
< Connection: keep-alive
< x-amz-id-2: im+nMggFzCCp+/mfLdWDr/ZwJFs=
< x-amz-request-id: 3B86E54874009D73
< ETag: "8d04fbe9d623bf03bb828594ca272063"
<
* Connection #0 to host 192.168.99.131 left intact

这会上传一个大小为320356字节的部分文件，原始文件为825210字节。

PUT更大的文件并调整标题：

[vagrant@gripper nginx]$ curl -vvvX PUT -H "Content-Length: 825210" -d @1408133479717.jpg http://192.168.99.131:8000/a.jpg
* About to connect() to 192.168.99.131 port 8000 (#0)
*   Trying 192.168.99.131...
* Connected to 192.168.99.131 (192.168.99.131) port 8000 (#0)
> PUT /a.jpg HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.99.131:8000
> Accept: */*
> Content-Length: 825210
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* Empty reply from server
* Connection #0 to host 192.168.99.131 left intact
curl: (52) Empty reply from server

我的问题是：

如何使用curl上传文件大小正确的文件？

Answer 1

听起来您可能遇到client_max_body_size的任何问题。就是说，默认值为1MB，所以令您惊讶的是825KB上传失败，除非您在nginx conf中修改了该值。

http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size

How to edit nginx.conf to increase file size upload

向AWS S3代理PUT HTTP请求失败

1 个答案: