java crawler:httpClient发起请求时出现ClientProtocolException错误

时间:2018-01-19 06:30:51

标签: java web-crawler httpclient

我想在模拟登录网站后获取页面数据。并且发生异常。

的问题:

java org.apache.http.client.ClientProtocolException at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at spider.SimulatePALogin.afterLoginPager(SimulatePALogin.java:92) at spider.SimulatePALogin.main(SimulatePALogin.java:41) Caused by: org.apache.http.ProtocolException: Content-Length header already present at org.apache.http.protocol.RequestContent.process(RequestContent.java:96) at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:182) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ... 4 more

```的java

    HttpUriRequest httpUriRequest = new HttpPost(
            "https://icore-pts.pingan.com.cn/ebusiness/login");
        // ?Accept?: application/json, text/javascript, */*; q=0.01
    httpUriRequest.setHeader("Host","pacas-login.pingan.com.cn");
    ......// set headers here 
    httpUriRequest.setHeader("Upgrade-Insecure-Requests","1");
    httpUriRequest
            .setHeader(
                    "User-Agent",
                    "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.89 Safari/537.36");

    // the ClientProtocolException occurs here
    HttpResponse response = HttpUtils.httpClient.execute(httpUriRequest);

```

我该如何解决?

提前thx!

1 个答案:

答案 0 :(得分:0)

我很难不查看错误信息。

我设置了“内容长度”标题,这使得异常发生。