Question

所以，我想从aspx页面获取数据。我需要执行以下步骤

1）发送GET请求并获取__VIEWSTATE和__EVENTVALIDATION。

2）发送带参数的POST请求（登录）然后更新__VIEWSTATE和__EVENTVALIDATION

3）发送带有参数的POST请求（选择字段），然后更新变量

4）发送带有参数的POST请求（按按钮）

5）解析页面内容。

页面内容在脚本后动态加载。我试着使用HttpUrlConnection

 public String sendGet(String url) throws Exception {
        StringBuilder result = new StringBuilder();
        URL ur = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) ur.openConnection();
        connection.setRequestMethod("GET");
        connection.setUseCaches(false);
        connection.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36");
        connection.setRequestProperty("Accept",
                "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
        connection.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
        BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String inputLine;
        while ((inputLine = br.readLine()) != null) {
            result.append(inputLine);
        }
        br.close();

        return result.toString();
    }
 public String sendPost(String url, List<NameValuePair> formParams) throws Exception {
        URL ur = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) ur.openConnection();
        connection.setReadTimeout(10000);
        connection.setDoOutput(true);
        connection.setDoInput(true);
        connection.setConnectTimeout(10000);
        connection.setRequestMethod("POST");
        for (HttpCookie cookie : this.cookies) {
            connection.addRequestProperty("Cookie", cookie.toString());
        }
        connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36");
        connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
        connection.setRequestProperty("Content-Language", "en-US");
        connection.connect();
        OutputStream os = connection.getOutputStream();
        BufferedWriter writer = new BufferedWriter(
                new OutputStreamWriter(os, "UTF-8"));
        writer.write(getQuery(formParams));
        writer.flush();
        writer.close();
        os.close();
        BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String response = "";
        String line;
        while ((line = br.readLine()) != null) {
            response += line;
        }
        System.out.println(Jsoup.parse(response).text());
        updateViewState(response);
        updateSubSession(response);
        return response;
    }

我留在第二步。 POST和GET subSession是不同的，但Cookie是相同的:)。因此，EVENTVALIDATION和VIEWSTATE无效，我再次获得“login.aspx”页面。这就是我获取Cookie的方法

List<HttpCookie> cookies;
public void init() throws Exception {
        CookieManager manager = new CookieManager();
        manager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
        CookieHandler.setDefault(manager);
        String html = sendGet(url);
        updateViewState(html);
        updateSubSession(html);
        CookieStore cookieJar = manager.getCookieStore();
        cookies=cookieJar.getCookies();
    }

所以，我尝试用subSession发送POST。

public String sendPost(String url, List<NameValuePair> formParams) throws Exception {
        URL ur = new URL(url+subSession);
        HttpURLConnection connection = (HttpURLConnection) ur.openConnection();

和...

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 500 for URL: http://www.aogc2.state.ar.us:8080/DWClient/Login.aspx?DWSubSession=9852&v=1589
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
    at com.company.UrlConnection.sendPost(UrlConnection.java:111)
    at com.company.UrlConnection.login(UrlConnection.java:138)
    at com.company.UrlConnection.start(UrlConnection.java:35)
    at com.company.Main.main(Main.java:11)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at

我认为获取cookie是错的，我试图像这样得到cookie：

connection.getHeaderFields().get("Set-Cookie");

但Cookie为null。哪里错了？非常感谢你！

HttpUrlConnection从aspx获取内容？

0 个答案: