Question

解析网页时，我得到链接href = http://www.onvista.de/aktien/snapshot.html?ID_OSI=36714349 在我的浏览器中发布此链接时，它会将其替换为“http://www.onvista.de/aktien/Adidas-Aktie-DE000A1EWWW0”并正确呈现。但是使用java我无法检索页面。我使用了以下建议的示例来显示重定向的URL。

import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;

public class GetRedirected {

    public GetRedirected() throws MalformedURLException, IOException {
        String url="http://www.onvista.de/aktien/snapshot.html?ID_OSI=36714349";
        URLConnection con = new URL( url ).openConnection();
        System.out.println( "orignal url: " + con.getURL() );
        con.connect();
        System.out.println( "connected url: " + con.getURL() );
        InputStream is = con.getInputStream();
        System.out.println( "redirected url: " + con.getURL() );
        is.close();
    }

    public static void main(String[] args) throws Exception {
        new GetRedirected();
    }
}

但它在“InputStream is =” - 语句中失败并附带错误消息。我该怎么解决这个问题。欢迎任何想法。

orignal url：www.onvista.de/aktien/snapshot.html？ID_OSI = 36714349

已连接的网址：www.onvista.de/aktien/snapshot.html？ID_OSI = 36714349

线程“main”中的异常java.io.IOException：服务器返回HTTP

响应代码：403 for URL：www.onvista.de/aktien/snapshot.html？ID_OSI = 36714349

at sun.net.www.protocol.http.HttpURLConnection.getInputStream（Unknown Source）

at de.gombers.broker ....

Answer 1

非常常见的错误：当HttpURLConnection的响应的HTTP状态代码指示错误（AFAIK＆gt; = 400）时，访问getInputStream()会引发异常。您必须检查getResponseCode()，然后决定是否必须致电getInputStream()或getErrorStream()。因此，您应该先致电getInputStream()。

，而不是致电getResponseCode()

但实际上我无法重现你的错误，对我而言它是有效的（虽然我使用了一个名为DavidWebb的小抽象库：

public void testAktienAdidas() throws Exception {

    Webb webb = Webb.create();
    Response<String> response = webb
            .get("http://www.onvista.de/aktien/snapshot.html?ID_OSI=36714349")
            .asString();

    assertEquals(200, response.getStatusCode());
    assertNotNull(response.getBody());
    assertTrue(response.getBody().contains("<!DOCTYPE html>"));
}

我没有获得重定向，可能这是通过JavaScript在客户端完成的，或者有一些服务器端逻辑可以评估像User-Agent这样的HTTP标头。

但如果您遇到重定向，可以告诉HttpURLConnection to automatically follow them：

conn.setInstanceFollowRedirects(true);

Answer 2

you can get retrieve it by this code
package Test;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class HttpRedirectExample {

  public static void main(String[] args) {

    try {

    String url = "http://www.onvista.de/aktien/snapshot.html?ID_OSI=36714349";
//  String urlTest="https://api.twitter.com/oauth/authenticate";

URL obj = new URL(url);
    HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
    conn.setReadTimeout(5000);
    conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
    conn.addRequestProperty("User-Agent", "Mozilla");
    conn.addRequestProperty("Referer", "google.com");

    System.out.println("Request URL ... " + url);

    boolean redirect = false;


    int status = conn.getResponseCode();
    if (status != HttpURLConnection.HTTP_OK) {
        if (status == HttpURLConnection.HTTP_MOVED_TEMP
            || status == HttpURLConnection.HTTP_MOVED_PERM
                || status == HttpURLConnection.HTTP_SEE_OTHER)
        redirect = true;
    }

    System.out.println("Response Code ... " + status);

    if (redirect) {

        // get redirect url from "location" header field
        String newUrl = conn.getHeaderField("Location");

        // get the cookie if need, for login
        String cookies = conn.getHeaderField("Set-Cookie");

        // open the new connnection again
        conn = (HttpURLConnection) new URL(newUrl).openConnection();
        conn.setRequestProperty("Cookie", cookies);
        conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
        conn.addRequestProperty("User-Agent", "Mozilla");
        conn.addRequestProperty("Referer", "google.com");

        System.out.println("Redirect to URL : " + newUrl);

    }

    BufferedReader in = new BufferedReader(
                              new InputStreamReader(conn.getInputStream()));
    String inputLine;
    StringBuffer html = new StringBuffer();

    while ((inputLine = in.readLine()) != null) {
        html.append(inputLine);
    }
    in.close();

    System.out.println("URL Content... \n" + html.toString());
    System.out.println("Done");

    } catch (Exception e) {
    e.printStackTrace();
    }

  }

}

如何通过java代码获取替换/重定向的URL

2 个答案: