Java .getInputStream()openConnection()HTTP响应代码错误

时间:2015-04-15 15:46:57

标签: java eclipse windows scraper

我正在尝试执行以下操作(在Java中):

  • 连接到某个代理服务器& http_url一些

但是我遇到了一些错误: java.net.ConnectException: Connection timed out: connect...

或与HTTP响应代码相关的错误:302, 400, FileNotFound, file server error等。

在我做的一些更改中,我甚至获得了200代码。 (当我只使用openConnection() =>(没有代理IP地址)时。 那是我最好的追踪。

我从IDE Eclipse Luna控制台的错误消息中获得了所有类:(未知来源)。

部分错误来自表单/或来自:.getInputStream()方法,我不知道是否有关于setDoInput()setDoOutput,编码或其他任何内容:< / p>

有些人可以帮助我吗?

这是我的代码:

url = new URL(http_url);
HttpURLConnection conn;
    try {
        conn = (HttpURLConnection)url.openConnection(proxy);
        conn.setRequestMethod("GET");
        conn.setRequestProperty("User-Agent", USERAGENT);

        conn.setUseCaches(false);
        conn.setRequestProperty("Accept", "*/*");
        conn.addRequestProperty("Referer", "http://www.google.com/");
        conn.setRequestProperty("Accept-Encoding", "gzip,deflate,sdch");
        conn.setRequestProperty("Accept-Language", "en-US,en;q=0.8");
        conn.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
        conn.setRequestProperty("X-Requested-With", "XMLHttpRequest");
        conn.setDoInput(true);
        System.out.println("response msg  " + conn.getResponseMessage() + " CODE");

        System.out.println("errorStream msg " + conn.getErrorStream());
        System.out.println("inputStream msg " + conn.getInputStream());
        String header_date = conn.getHeaderField("Date");
        System.out.println(" date es: " + header_date);
        String line = null;
        StringBuffer tmp = new StringBuffer();
        System.out.println("the code is :" + conn.getResponseCode());

        if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {
            BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));

            while ((line = in.readLine()) != null) {
                tmp.append(line);
            }
            System.out.println("value line is:  " + line +"& date is: " + header_date);

            Scrape(String.valueOf(tmp)); // temp.toString()

            in.close();
            in = null;
            url = null;
            conn.disconnect();
            conn = null;
        } else {   
            System.out.println("something bad happened code <>200, debug from your server"); 
        }        
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();    
    } catch (MalformedURLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (UnsupportedEncodingException e1) {
        // TODO Auto-generated catch block
        e1.printStackTrace();
    }

1 个答案:

答案 0 :(得分:0)

要解决您的代理问题,您可以尝试使用Proxy,如下所示

Proxy proxy= new Proxy(Proxy.Type.HTTP, new InetSocketAddress(<Proxy IP Address as String>, <Proxy Port Number as Integer>));
        HttpURLConnection http_conn=(HttpURLConnection)request_url.openConnection(proxy);