Question

我正在构建一个webcrawler，并且有一种检查错误链接的方法。有一次，我试图获取HTTP响应代码以确定它是否有效。尽管给它一个有效的URL（在浏览器中打开它就好了）它仍然返回它不是有效的。这是代码：

public static boolean isBrokenLink(URL baseURL, String theHREF) {
        boolean isBroken = false;
        if (baseURL == null) {
            try {
                baseURL = new URL("HTTP", "cs.uwec.edu/~stevende/cs145testpages/", theHREF);
                System.out.println(baseURL);
            } catch (MalformedURLException e) {
                isBroken = true;
                //e.printStackTrace();
            }
        }
        try {
            URLConnection con = baseURL.openConnection();
            HttpURLConnection httpProtocol = (HttpURLConnection) con;
            System.out.println(httpProtocol.getResponseCode());
            if (httpProtocol.getResponseCode() != 200 && httpProtocol.getResponseCode() == -1) {
                isBroken = true;
            }
        } catch (IOException e) {
            isBroken = true;
            e.printStackTrace();
        }

        return isBroken;
    }   
            }

here是我传递的网址。 isBroken是返回的布尔值。我将baseURL作为null传递，将HREF作为相对链接传递（page2.htm）。我在从字符串创建URL后打印出来的URL。谢谢你的帮助！这是错误：

java.net.UnknownHostException: cs.uwec.edu/~stevende/cs145testpages/
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at java.net.Socket.connect(Socket.java:528)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
    at sun.net.www.http.HttpClient.New(HttpClient.java:308)
    at sun.net.www.http.HttpClient.New(HttpClient.java:326)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
    at edu.uwec.cs.carpenne.webcrawler.Webcrawler.isBrokenLink(Webcrawler.java:106)
    at edu.uwec.cs.carpenne.webcrawler.Webcrawler.main(Webcrawler.java:181)

Answer 1

异常告诉我们，它使用主机名和本地部分作为（未知）主机。这看起来像你错误地构造了URL。也许你忘了使用http://前缀或使用了错误的getter？您可以通过调用baseURL.getHost()，baseURL.getPath()和baseURL.getProtocol()来调试它，看看它是否会返回cs.uwec.edu和/~steve...以及http。

我刚刚注意到您添加了带有new URL("HTTP", "cs.uwec.edu/~stevende/cs145testpages/", theHREF)的baseURL，这是错误的，您需要使用new URL("http", "cs.uwec.edu", 80, "/~stevende/cs145testpages/#"+theHREF)。但是，您通常可以跳过anchor / ref，因为它不会传输到服务器。

您还可以使用单个参数构造函数new URL("http://cs.uwec.edu//~stevende/cs145testpages/")。

.getResponse代码在有效URL上抛出IOException

1 个答案: