Question

我正在使用Jetty的HttpClient构建一个简单的代理服务器。我在Java 1.8.0_45上使用Jetty版本9.3.10.v20160621。

我有一种情况，我在资源上执行GET，它将返回大小约为3.5M的响应（我使用soapUI确定了这一点）。知道Jetty的默认最大响应大小是2M，我在创建HttpClient实例时会执行以下操作。

        HttpClient client = new HttpClient();
        client.setResponseBufferSize(4194304);
        client.start();

稍后我执行同步GET请求，如下所示：

        System.out.println("response buffer size = " + client.getResponseBufferSize());
        retVal = client.GET(uri);

控制台日志包含以下内容：

响应缓冲区大小= 4194304

然而，当我执行GET（）时，我得到一个java.util.concurrent.ExecutionException（下面的堆栈跟踪）。我只能得出结论：（a）setResponseBufferSize（）方法中存在错误，或者（b）setResponseBufferSize（）方法没有做文档所说的（设置响应缓冲区大小）。有人知道这是怎么回事吗？

java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Buffering capacity exceeded
    at org.eclipse.jetty.client.util.FutureResponseListener.getResult(FutureResponseListener.java:118)
    at org.eclipse.jetty.client.util.FutureResponseListener.get(FutureResponseListener.java:101)
    at org.eclipse.jetty.client.HttpRequest.send(HttpRequest.java:652)
    at org.eclipse.jetty.client.HttpClient.GET(HttpClient.java:343)
    at oracle.paas.tools.sifter.proxy.ProxySession.get(ProxySession.java:106)
    at oracle.paas.tools.sifter.proxy.Endpoint.doGet(Endpoint.java:75)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:837)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:524)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Buffering capacity exceeded
    at org.eclipse.jetty.client.util.BufferingResponseListener.onContent(BufferingResponseListener.java:114)
    at org.eclipse.jetty.client.api.Response$Listener$Adapter.onContent(Response.java:248)
    at org.eclipse.jetty.client.ResponseNotifier.notifyContent(ResponseNotifier.java:124)
    at org.eclipse.jetty.client.ResponseNotifier.access$100(ResponseNotifier.java:35)
    at org.eclipse.jetty.client.ResponseNotifier$ContentCallback.process(ResponseNotifier.java:272)
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
    at org.eclipse.jetty.client.ResponseNotifier.notifyContent(ResponseNotifier.java:117)
    at org.eclipse.jetty.client.HttpReceiver.responseContent(HttpReceiver.java:326)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.content(HttpReceiverOverHTTP.java:256)
    at org.eclipse.jetty.http.HttpParser.parseContent(HttpParser.java:1584)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1332)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.parse(HttpReceiverOverHTTP.java:158)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:119)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69)
    at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:90)
    at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:115)
    ... 9 more

Answer 1

HttpClient.setResponseBufferSize(int)用于配置用于从网络读取单个缓冲区的内部缓冲区大小。

它与您的响应正文内容大小无关。

您正在处理的情况是，简单的HttpClient.GET(uri)调用会将响应缓冲到FutureResponseListener，其内存使用量限制为2MB。

此限制仅仅是java Future概念的副作用。

这是一种速记/便利方法，适用于小响应。

您的响应更大，因此您可以使用HttpClient的异步功能从HttpClient管理的响应内容缓冲区/流中读取数据，根据您的需要处理数据。

示例：

    // In Initialization Code, start the client
    HttpClient client = new HttpClient();
    client.start();

    // In code much later one, use the client
    // Don't constantly start/stop the HttpClient
    // Treat the HttpClient as a browser, and each newRequest() as
    //   a tab on that browser.
    InputStreamResponseListener listener = new InputStreamResponseListener();
    // Send asynchronously with the InputStreamResponseListener
    client.newRequest(uri).send(listener);

    // Call to the listener's get() blocks until the headers arrived
    Response response = listener.get(5, TimeUnit.SECONDS);

    // Now check the response information that arrived to decide whether to read the content
    if (response.getStatus() == 200)
    {
        byte[] buffer = new byte[256];
        try (InputStream input = listener.getInputStream())
        {
            while (true)
            {
                int read = input.read(buffer);
                if (read < 0)
                    break;
                // Do something with the bytes just read
            }
        }
    }
    else
    {
        response.abort(new Exception());
    }

查看HttpClient Usage.java以获取更多示例

Answer 2

我找到了解决问题的另一种方法。这个解决方案不像Joakim的解决方案那么通用。如果响应大于4 MiB（或您指定的任何内容），它将获得相同的异常。另一方面，它的代码更少。

Request request = client.newRequest(uri);
FutureResponseListener listener = new FutureResponseListener(request, 4 * 1024 * 1024);
request.send(listener);
retVal = listener.get();

Jetty的httpClient.setResponseBufferSize（）方法可以做什么吗？

2 个答案: