Question

我在使用Apache HttpClient4通过POST操作读取大型响应流时遇到问题。响应已发送，但长度未知。当我使用curl进行使用时，它立即开始被使用，但是它开始使用之前的HttpClient，直到它收到整个响应为止。

使用curl时，活动如下所示，并且几乎立即开始写入输出：

$ curl -v -X POST --data-binary @input.gz http://chemservices:8080/chem-services-cdk-basic/rest/v1/converters/dataset_to_sdf  -H 'Content-Type: application/x-squonk-dataset-molecule+json' -H 'Content-Encoding: gzip' -H 'Accept: chemical/x-mdl-sdfile' -o output.sdf -v
Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 172.18.0.6...
* TCP_NODELAY set
* Connected to chemservices (172.18.0.6) port 8080 (#0)
> POST /chem-services-cdk-basic/rest/v1/converters/dataset_to_sdf HTTP/1.1
> Host: chemservices:8080
> User-Agent: curl/7.58.0
> Content-Type: application/x-squonk-dataset-molecule+json
> Content-Encoding: gzip
> Accept: chemical/x-mdl-sdfile
> Content-Length: 19397182
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
} [16384 bytes data]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, HEAD, POST, PUT, DELETE, TRACE, OPTIONS, CONNECT, PATCH
< Access-Control-Max-Age: 3600
< Access-Control-Allow-Headers: Origin, Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers
< Content-Type: chemical/x-mdl-sdfile
< Transfer-Encoding: chunked
< Date: Sun, 13 Jan 2019 12:30:18 GMT
< 
{ [1031 bytes data]
100  172M    0  154M  100 18.4M  7103k   851k  0:00:22  0:00:22 --:--:-- 7301k
* Connection #0 to host chemservices left intact

但是当我使用HttpClient时，execute（）操作将阻塞，直到整个响应都被写入为止：

LOG.info("Posting commencing");
CloseableHttpResponse resp = httpclient.execute(httpPost);
LOG.info("Posting complete");

正在根据请求设置Content-Type，Content-Encoding，Accept和Accept-Encoding标头。

如果我将SocketTimeout参数设置为足够大的值，我可以获得响应，但是显然这不是正确的解决方案！

关于如何正确处理此问题的任何建议？

Answer 1

我相信您将需要使用HttpAsyncClient来实现异步请求处理。

考虑以下示例，它将以分块形式传输响应，您可以在覆盖的<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script> <div class="container"> <div class="item play">1</div> <div class="item do-not-play">2</div> <div class="item play">3</div> <div class="item play">4</div> <div class="item do-not-play">5</div> <div class="item do-not-play">6</div> <div class="item play">7</div> </div>方法中处理该响应。出于本示例的目的，我只是让它打印每个块的长度。

显然，您将需要修改端点，标头并根据需要请求数据：

alpha = 0.5
lam = cvx.Paramter(nonneg=True)
lam.value = 1e2

直到数据完成才返回响应

1 个答案: