我编写了一个JAVA抓取工具,尝试使用proxy
并忽略任何https certification
。
但它不适用于
java.lang.UnsupportedOperationException (at org.apache.http.impl.client.InternalHttpClient.getParams)
我搜索的解决方案主要是说我的HttpClient
版本已经过时了,但是我从apache网站更新到最新版本,这种情况仍然存在。
以下代码是我的抓取代码:
public static void main(String[] args) {
try{
TrustManager[] trustAllCerts = new TrustManager[] {
new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {}
public void checkServerTrusted(X509Certificate[] certs, String authType) {}
}
};
SSLContext ctx = SSLContext.getInstance("TLS");
ctx.init(null, trustAllCerts, null);
LayeredConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(ctx);
CloseableHttpClient client = HttpClients.custom().setSSLSocketFactory(sslSocketFactory).build();
HttpHost proxy = new HttpHost("127.0.0.1", 8888,"http");
client.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY,proxy);
HttpGet request = new HttpGet("https://www.javaworld.com.tw/jute/post/view?bid=29&id=312144");
CloseableHttpResponse response = client.execute(request);
String entity = EntityUtils.toString(response.getEntity(), "utf-8");
System.out.println(entity);
}catch (Exception e) {
e.printStackTrace();
}
}
非常感谢任何解决方案。
答案 0 :(得分:0)
我遇到了同样的问题,因为该方法已弃用,在最新版本中不可用。
我尝试了以下代码,它对我有用
public static HttpClient createClient() {
try {
SSLContextBuilder builder = new SSLContextBuilder();
builder.useProtocol("TLSv1.2");
builder.loadTrustMaterial(null, new TrustStrategy() {
@Override
public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException {
return true;
}
});
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
builder.build());
HttpClientBuilder hcBuilder = HttpClients.custom();
HttpHost httpProxy = new HttpHost(bundle.getString("PROXY_HOST"), Integer.parseInt(bundle.getString("PROXY_PORT")));
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(httpProxy);
hcBuilder.setRoutePlanner(routePlanner);
CloseableHttpClient httpclient = hcBuilder
.setSSLSocketFactory(sslsf).build();
return httpclient;
} catch (Exception e) {
throw new RuntimeException(e);
}
}