HttpClient连接在数据传输期间关闭

时间:2013-06-20 11:59:14

标签: java httpclient apache-commons-httpclient

从服务器获取xml时,我遇到了一个问题,其中HttpClient GET方法提供的响应xml是部分的。而使用浏览器的xml按预期填满。

相同的代码是

        InputStream response = null;
        HttpClient client = new HttpClient();
        method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, new DefaultHttpMethodRetryHandler(3, false));

        int statusCode = client.executeMethod(method);

        if (statusCode != HttpStatus.SC_OK) {
        System.err.println("Method failed: " + method.getStatusLine());
        }

        response = method.getResponseBodyAsStream();

以下是httpclient调试日志的输出

        16:07:24(GetMethod.java:89) DEBUG org.apache.commons.httpclient.methods.GetMethod - enter GetMethod(String)
        16:07:26(DefaultHttpParams.java:150) DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set parameter http.method.retry-handler = org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@578cef18
        16:07:29(HttpClient.java:322) DEBUG org.apache.commons.httpclient.HttpClient - enter HttpClient.executeMethod(HttpMethod)
        16:07:29(HttpClient.java:373) DEBUG org.apache.commons.httpclient.HttpClient - enter HttpClient.executeMethod(HostConfiguration,HttpMethod,HttpState)
        16:07:29(HttpMethodDirector.java:378) DEBUG org.apache.commons.httpclient.HttpMethodDirector - Attempt number 1 to process request
        16:07:29(HttpConnection.java:684) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.open()
        16:07:29(HttpConnection.java:691) DEBUG org.apache.commons.httpclient.HttpConnection - Open connection to mysite:80
        16:07:29(HttpMethodBase.java:976) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.execute(HttpState, HttpConnection)
        16:07:29(HttpMethodBase.java:1916) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.writeRequest(HttpState, HttpConnection)
        16:07:29(HttpMethodBase.java:2071) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.writeRequestLine(HttpState, HttpConnection)
        16:07:29(HttpMethodBase.java:1379) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.generateRequestLine(HttpConnection, String, String, String, String)
        16:07:29(Wire.java:69) DEBUG httpclient.wire.header - >> "GET /mypath HTTP/1.1[\r][\n]"
        16:07:29(HttpConnection.java:1031) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.print(String)
        16:07:29(HttpConnection.java:941) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[])
        16:07:29(HttpConnection.java:962) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[], int, int)
        16:07:29(HttpMethodBase.java:2034) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.writeRequestHeaders(HttpState,HttpConnection)
        16:07:29(HttpMethodBase.java:1300) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.addRequestHeaders(HttpState, HttpConnection)
        16:07:29(HttpMethodBase.java:1325) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.addUserAgentRequestHeaders(HttpState, HttpConnection)
        16:07:29(HttpMethodBase.java:1211) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.addHostRequestHeader(HttpState, HttpConnection)
        16:07:29(HttpMethodBase.java:1235) DEBUG org.apache.commons.httpclient.HttpMethodBase - Adding Host request header
        16:07:29(HttpMethodBase.java:1163) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.addCookieRequestHeader(HttpState, HttpConnection)
        16:07:29(HttpState.java:172) DEBUG org.apache.commons.httpclient.HttpState - enter HttpState.getCookies()
        16:07:29(CookieSpecBase.java:548) DEBUG org.apache.commons.httpclient.cookie.CookieSpec - enter CookieSpecBase.match(String, int, String, boolean, Cookie[])
        16:07:29(HttpMethodBase.java:1262) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.addProxyConnectionHeader(HttpState, HttpConnection)
        16:07:29(Wire.java:69) DEBUG httpclient.wire.header - >> "User-Agent: Jakarta Commons-HttpClient/3.0.1[\r][\n]"
        16:07:29(HttpConnection.java:1031) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.print(String)
        16:07:29(HttpConnection.java:941) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[])
        16:07:29(HttpConnection.java:962) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[], int, int)
        16:07:29(Wire.java:69) DEBUG httpclient.wire.header - >> "Host: mysite[\r][\n]"
        16:07:29(HttpConnection.java:1031) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.print(String)
        16:07:29(HttpConnection.java:941) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[])
        16:07:29(HttpConnection.java:962) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[], int, int)
        16:07:29(HttpConnection.java:1000) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.writeLine()
        16:07:29(HttpConnection.java:941) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[])
        16:07:29(HttpConnection.java:962) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.write(byte[], int, int)
        16:07:29(Wire.java:69) DEBUG httpclient.wire.header - >> "[\r][\n]"
        16:07:29(HttpConnection.java:825) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.flushRequestOutputStream()
        16:07:29(HttpMethodBase.java:1585) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.readResponse(HttpState, HttpConnection)
        16:07:29(HttpMethodBase.java:1823) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.readStatusLine(HttpState, HttpConnection)
        16:07:29(HttpConnection.java:1112) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.readLine()
        16:07:29(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:07:29(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:07:29(Wire.java:69) DEBUG httpclient.wire.header - << "HTTP/1.0 200 OK[\r][\n]"
        16:09:31(HttpMethodBase.java:1785) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.readResponseHeaders(HttpState,HttpConnection)
        16:09:31(HttpConnection.java:856) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.getResponseInputStream()
        16:09:31(HttpParser.java:159) DEBUG org.apache.commons.httpclient.HttpParser - enter HeaderParser.parseHeaders(InputStream, String)
        16:09:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:09:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:09:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:09:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:09:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(HttpParser.java:104) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readLine(InputStream, String)
        16:11:31(HttpParser.java:73) DEBUG org.apache.commons.httpclient.HttpParser - enter HttpParser.readRawLine()
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "X-DGKeepAlive: on[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "Date: Thu, 10:37:29 GMT[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "Server: Apache[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "Set-Cookie: JSESSIONID=1DA9A4C37BD6883B837BA8725E3CCC5D; Path=/mypath[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "Vary: Accept-Encoding[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "Content-Type: text/xml;charset=ISO-8859-1[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "X-Cache: MISS from ajay[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "X-Cache-Lookup: MISS from ajay:3128[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "Via: 1.0 ajay:3128 (squid/2.6.STABLE21)[\r][\n]"
        16:11:31(Wire.java:69) DEBUG httpclient.wire.header - << "Proxy-Connection: close[\r][\n]"
        16:11:31(HttpMethodBase.java:1462) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.processResponseHeaders(HttpState, HttpConnection)
        16:11:31(CookieSpecBase.java:242) DEBUG org.apache.commons.httpclient.cookie.CookieSpec - enter CookieSpecBase.parse(String, port, path, boolean, String)
        16:11:31(CookieSpecBase.java:112) DEBUG org.apache.commons.httpclient.cookie.CookieSpec - enter CookieSpecBase.parse(String, port, path, boolean, Header)
        16:11:31(HeaderElement.java:193) DEBUG org.apache.commons.httpclient.HeaderElement - enter HeaderElement.parseElements(char[])
        16:11:31(Cookie.java:108) DEBUG org.apache.commons.httpclient.Cookie - enter Cookie(String, String, String, String, Date, boolean)
        16:11:31(RFC2109Spec.java:130) DEBUG org.apache.commons.httpclient.cookie.CookieSpec - enter RFC2109Spec.validate(String, int, String, boolean, Cookie)
        16:11:31(CookieSpecBase.java:368) DEBUG org.apache.commons.httpclient.cookie.CookieSpec - enter CookieSpecBase.validate(String, port, path, boolean, Cookie)
        16:11:31(HttpState.java:124) DEBUG org.apache.commons.httpclient.HttpState - enter HttpState.addCookie(Cookie)
        16:11:31(RFC2109Spec.java:243) DEBUG org.apache.commons.httpclient.cookie.CookieSpec - enter RFC2109Spec.formatCookie(Cookie)
        16:11:31(HttpMethodBase.java:1506) DEBUG org.apache.commons.httpclient.HttpMethodBase - Cookie accepted: "$Version=0; JSESSIONID=1DA9A4C37BD6883B837BA8725E3CCC5D; $Path=/mypath"
        16:11:31(HttpMethodBase.java:1636) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.readResponseBody(HttpState, HttpConnection)
        16:11:31(HttpMethodBase.java:1671) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.readResponseBody(HttpConnection)
        16:11:31(HttpConnection.java:856) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.getResponseInputStream()
        16:11:31(HttpMethodBase.java:2140) DEBUG org.apache.commons.httpclient.HttpMethodBase - enter HttpMethodBase.canResponseHaveBody(int)
        16:11:31(Wire.java:83) DEBUG httpclient.wire.content - <<"#######INCOMPLETE XML#########"
        16:12:12(HttpMethodBase.java:921) DEBUG org.apache.commons.httpclient.HttpMethodBase - Resorting to protocol version default close connection policy
        16:12:12(HttpMethodBase.java:929) DEBUG org.apache.commons.httpclient.HttpMethodBase - Should close connection, using HTTP/1.0
        16:12:12(HttpConnection.java:1147) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.close()
        16:12:12(HttpConnection.java:1214) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.closeSockedAndStreams()
        16:12:12(HttpConnection.java:1173) DEBUG org.apache.commons.httpclient.HttpConnection - enter HttpConnection.releaseConnection()
        16:12:12(HttpConnection.java:1177) DEBUG org.apache.commons.httpclient.HttpConnection - Releasing connection back to connection manager.

当通过浏览器获取相同的网址时,它可以正常工作。请求的标题如下 浏览器响应标题查看源

        Content-Encoding    gzip
        Content-Type    text/xml;charset=ISO-8859-1
        Date    Thu, 20 Jun 2013 11:24:54 GMT
        Proxy-Connection    close
        Server  Apache
        Vary    Accept-Encoding
        Via 1.0 ajay:3128 (squid/2.6.STABLE21)
        X-Cache MISS from ajay
        X-Cache-Lookup  MISS from ajay:3128

浏览器请求标题浏览源

        Accept  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
        Accept-Encoding gzip, deflate
        Accept-Language en-US,en;q=0.5
        Connection  keep-alive
        Cookie  JSESSIONID=6265E9445BB5CB5EF2594A9BEC19B119
        Host    mysite
        User-Agent  Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0

任何人都可以在httpclient调用代码或其他服务器端解释问题的位置以及如何纠正这个问题? 提前谢谢。

解析响应时抛出异常

       DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(inputStreamOfResponse);

例外是

    [Fatal Error] :2:1: XML document structures must start and end within the same entity.
    org.xml.sax.SAXParseException; lineNumber: 2; columnNumber: 1; XML document structures must start and end within the same entity.

1 个答案:

答案 0 :(得分:0)

终于解决了这个问题,设置以下标题为我工作。

method.setRequestHeader("Accept-Encoding", "gzip, deflate");

postResponse = IOUtils.toString(new GZIPInputStream(method.getResponseBodyAsStream()));