我正在尝试通过代理抓取网站,我找到了一些免费代理。他们在我的win7中表现不错,但我不知道如何在java中的httpclient中设置代理。
这是我的代码:
HttpClient httpClient = new HttpClient();
httpClient.getHostConfiguration().setProxy("us1.vpnbook.com",8080);
httpClient.getParams().setAuthenticationPreemptive(true);
httpClient.getState().setProxyCredentials(AuthScope.ANY, new UsernamePasswordCredentials("vpn","Phavupr7"));
HttpMethod method = new GetMethod("http://www.google.com");
try {
httpClient.executeMethod(method);
if (method.getStatusCode() == HttpStatus.SC_OK) {
String response = method.getResponseBodyAsString();
System.out.println("Response = " + response);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
method.releaseConnection();
}
September 11, 2014 4:10:10 PM org.apache.commons.httpclient.HttpMethodDirector authenticateHost
Warning: Required credentials not available for BASIC <any realm>@www.google.com:80
September 11, 2014 4:10:10 PM org.apache.commons.httpclient.HttpMethodDirector authenticateHost
Warning: Preemptive authentication requested but no default credentials available
September 11, 2014 4:10:31 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Information: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
September 11, 2014 4:10:31 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Information: Retrying request
September 11, 2014 4:10:52 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Information: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
September 11, 2014 4:10:52 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Information: Retrying request
September 11, 2014 4:11:13 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Information: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
September 11, 2014 4:11:13 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Information: Retrying request
java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at web2.h.main(h.java:73)