Question

我正在尝试使用我编写的Java程序从站点获取信息。

当我使用selenium时，一切正常，但当我尝试使用prowser或httpurlconnection时，该网站会屏蔽我的IP。

我正在将用户代理设置为Chrome/44.0.2403.155，以及原始网站请求中的其他参数（我是在HTTPFox的帮助下获得的。）

我正在处理Cookie（我从FirefoxWebDriver获得的Cookie），但结果总是一样的：我的IP已被阻止。

我得到了我需要六次的信息，然后网站阻止了我的IP。当我不使用cookies（“退出模式”）时，一切正常。

Answer 1

您访问的网站是通过HTTPS还是HTTP？

请尝试将打印GET请求的代码发送给网站。

public class ConnectToHTTPS {
    public static void main(String[] args) throws Exception {
        // Create a trust manager that does not validate certificate chains
        TrustManager[] trustAllCerts = new TrustManager[] {new X509TrustManager() {
                public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                    return null;
                }
                public void checkClientTrusted(X509Certificate[] certs, String authType) {
                }
                public void checkServerTrusted(X509Certificate[] certs, String authType) {
                }
            }
        };

        // Install the all-trusting trust manager
        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

        // Create all-trusting host name verifier
        HostnameVerifier allHostsValid = new HostnameVerifier() {
            public boolean verify(String hostname, SSLSession session) {
                return true;
            }
        };
        // Install the all-trusting host verifier
        HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);


        URL url = new URL("YOUR SITE ADDRESS");
        HttpURLConnection cox= (HttpURLConnection) url.openConnection();      

       // URLConnection con = url.openConnection();

        Reader reader = new InputStreamReader(cox.getInputStream());
        while (true) {
            int ch = reader.read();
            if (ch==-1) {
                break;
            }
            System.out.print((char)ch);
        }


    }
}

OR

只需尝试在Java程序中设置代理

    System.setProperty("http.proxyHost", "YOUR PROXY");

          URL url = new URL("YOUR WebSite");

            URLConnection con = url.openConnection();

            BufferedReader in = new BufferedReader(new InputStreamReader(
                    con.getInputStream()));

            String inputLine;
            while ((inputLine = in.readLine()) != null)
                System.out.println(inputLine);

            in.close();

    }

如果证书问题已解决，则为

You can use GET once the above worked 

 // Install the all-trusting host verifier
            HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

            String url = "http://www.google.com/search?q=mkyong";

            URL obj = new URL(url);
            HttpURLConnection con = (HttpURLConnection) obj.openConnection();

            // optional default is GET
            con.setRequestMethod("GET");

Java http请求在网站上被禁止

1 个答案: