目标服务器无法使用htmlunit和tor响应异常

时间:2012-07-02 08:11:06

标签: java htmlunit tor

我正在使用htmlunit向网站发送请求并寻求匿名。但是我得到了

目标服务器无法响应异常。我在google上搜索并找到了以下代码。

HttpMethodRetryHandler myretryhandler = new HttpMethodRetryHandler() {
    public boolean retryMethod(
        final HttpMethod method, 
        final IOException exception, 
        int executionCount) {
        if (executionCount >= 5) {
            // Do not retry if over max retry count
            return false;
        }
        if (exception instanceof NoHttpResponseException) {
            // Retry if the server dropped connection on us
            return true;
        }
        if (!method.isRequestSent()) {
        // Retry if the request has not been sent fully or
        // if it's OK to retry methods that have been sent
        return true;
    }
    // otherwise do not retry
    return false;
    }
};

GetMethod httpget = new GetMethod("http://www.whatever.com/");
httpget.getParams().
setParameter(HttpMethodParams.RETRY_HANDLER, myretryhandler);
try {
    client.executeMethod(httpget);
    System.out.println(httpget.getStatusLine().toString());
} finally {
    httpget.releaseConnection();
}

但是我在htmlunit中找不到如何做到这一点。我怎样才能做到这一点?

2 个答案:

答案 0 :(得分:2)

提供的代码是一个HttpClient示例,如果您想使用htmlunit,请检查他们的网站http://htmlunit.sourceforge.net/并使用这些代码段您应该能够发送(发布?)请求

WebClient client = new WebClient(BrowserVersion.FIREFOX_3_6);
client.setTimeout(60000);
client.setRedirectEnabled(true);
client.setJavaScriptEnabled(true);
client.setThrowExceptionOnFailingStatusCode(false);
client.setThrowExceptionOnScriptError(false);
client.setCssEnabled(false);
client.setUseInsecureSSL(true);

    HtmlPage page = null;
    try {
        page = client.getPage("http://www.whatever.com");
    } catch (Exception e) {
        // TODO Auto-generated catch block
    }
    if (page.getWebResponse().getStatusCode() == 404) {
        System.out.println("Page not found");
    }

    // Post a request
    WebRequest request = new WebRequest(new URL("http://www.whatever.com/post_url"));
    request.setHttpMethod(HttpMethod.POST);
    List<NameValuePair> params = new ArrayList<NameValuePair>();
    params.add(new NameValuePair("login", userLogin));
    params.add(new NameValuePair("pass", userPassword));
    request.setRequestParameters(params);

    page = client.getPage(request);

答案 1 :(得分:0)

    final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_45);
    webClient.getOptions().setTimeout(20000);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setUseInsecureSSL(true);
    webClient.getOptions().setRedirectEnabled(true);

    webClient.setConfirmHandler(new ConfirmHandler() {
        public boolean handleConfirm(Page page, String message) {
            return true;
        }
    });

    String binary = DatatypeConverter.printBase64Binary(auth.getBytes());
    webClient.addRequestHeader("Authorization", "Basic " + binary);

    //get General page
    final HtmlPage page = webClient.getPage("http://adress");