Question

我必须通过网络获取一些html页面，我正在使用http-components包中的apache httpclient。我已将connectionTimeout，soTimeout设置为5000毫秒以及将重定向转为false，但代码似乎在socketread函数中被阻止。

被阻止的特定网址如下： http://high.lrn.fm http://gotradioaac04.lbdns-streamguys.com

任何人都可以给我任何关于如何阻止线程阻止对httpclient的套接字读取操作的建议

我的代码在下面给出了参考资料

public class HTTPDataDownloader {

    private static final Logger logger          = Logger.getLogger(HTTPDataDownloader.class);
    private int                 soTimeout;                                                      // ms
    private int                 connTimeout;                                                    // ms
    private HttpParams          httpParameters;
    private HttpClient          httpClient;
    private static final String HTTP_CONTENT    = "text/html";

    public HTTPDataDownloader( int soTimeout, int connTimeout ) {
        this.soTimeout = soTimeout;
        this.connTimeout = connTimeout;
        initialize();
    }

    private void initialize() {
        httpParameters = new BasicHttpParams();
        // Set the timeout in milliseconds until a connection is established.
        HttpConnectionParams.setConnectionTimeout(httpParameters, connTimeout);
        // Set the default socket timeout (SO_TIMEOUT)
        // in milliseconds which is the timeout for waiting for data.
        HttpConnectionParams.setSoTimeout(httpParameters, soTimeout);
        HttpConnectionParams.setStaleCheckingEnabled(httpParameters, false);
        HttpConnectionParams.setLinger(httpParameters, 5);
        httpParameters.setParameter("http.protocol.handle-redirects",false);
        HttpClientParams.setRedirecting(httpParameters, false);
        //HttpClientParams.setConnectionManagerTimeout(httpParameters, 500);

        httpClient = new DefaultHttpClient(httpParameters);
        httpClient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
    }

    private void setRetryHandler() {
        HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(2, true);

        //httpClient.setHttpRequestRetryHandler(retryHandler);
        //httpClient.getParams().setParameter(HttpParams, arg1)
    }

    // takes the url, make the connection and fetch the data
    public String fetch( String urlname ) {
        urlname = formatURL(urlname);
        HttpGet httpget = new HttpGet();
        try {
            httpget.setURI(new URI(urlname));
        } catch ( URISyntaxException e ) {
            logger.error(e.toString());
            return null;
        }

        StringBuilder content = new StringBuilder();
        InputStream instream = null;
        try {
            HttpResponse httpResponse = httpClient.execute(httpget);
            HttpEntity entity = httpResponse.getEntity();
            if ( entity != null ) {

                String contentType = entity.getContentType().getValue();
                HeaderElementIterator it = new BasicHeaderElementIterator(
                                                                          httpResponse.headerIterator(HTTP.CONN_KEEP_ALIVE));
            /*  while (it.hasNext()) {
                    HeaderElement he = it.nextElement();
                    String param = he.getName(); 
                    String value = he.getValue();
                    if (value != null && param.equalsIgnoreCase("timeout")) {
                        System.out.println(value + urlname);
                    }
                }*/
                if ( contentType != null && contentType.indexOf(HTTP_CONTENT) >= 0 ) {
                    instream = entity.getContent();
                    BufferedReader br = new BufferedReader(new InputStreamReader(instream));
                    String line;
                    String newLine = System.getProperty("line.separator");

                    while ( (line = br.readLine()) != null )
                        content.append(line + newLine);
                    logger.info("Downloaded: " + httpget.getURI().toString());
                    return content.toString();
                }
            }
        }
        catch ( ClientProtocolException e ) {       
            logger.info("ClientProtocolException: " + e + " " + urlname);
        } catch ( ConnectTimeoutException e ) {
            logger.info("ConnectionTimeoutException: " + e + " " + urlname);
        } catch ( SocketTimeoutException e ) {
            logger.info("SocketTimeoutException: " + e + " " + urlname);
        } catch ( IOException e ) {
            logger.info("IOException: " + e + " " + urlname);
        } catch ( Exception e ) {
            logger.equals("Exception: " + e + " " + urlname);
        } finally {
            httpget.abort();    
            try {               
                if (instream != null)
                    instream.close(); 
            } catch ( IOException e ) { }
        }

        return null;
    }

Answer 1

我也遇到了这个问题，因为httpclient.execute线程没有释放连接。获得响应后，您可以尝试添加两行：

httpget.releaseConnection();
httpclient.getConnectionManager().shutdown();

Apache的httpclient阻塞在socketread上，即使设置了soTimeout

1 个答案: