我正致力于通过 HtmlUnit 抓取网站。我有2个下拉列表。 2个下拉列表已链接,因此选择下拉列表1 选项会对 onChange()进行JavaScript调用。根据此选择,将填充下拉列表2 中的值。
这是我写的代码: 假设我在select标签(id:ddldistlist)中选择“4-Medak”选项,那么它应该通过JS调用在select标签(id:ddlaclist)中填充它的相应选项值。但是,当我通过HtmlUnit进行此操作时,这种情况不会发生。
HTML code:
<select name="ddldistlist" onchange="javascript:setTimeout('__doPostBack(\'ddldistlist\',\'\')', 0)" id="ddldistlist" style="font-weight:bold;width:150px;">
<option value="-Select-">-Select-</option>
<option value="1">1-Adilabad</option>
<option value="2">2-Nizamabad</option>
<option value="3">3-Karimnagar</option>
<option value="4" selected="selected">4-Medak</option>
<option value="5">5-Rangareddy</option>
<option selected="selected" value="6">6-Hyderabad</option>
</select>
<select name="ddlaclist" id="ddlaclist" style="font-weight:bold;width:155px;">
<option value="-Select-">-Select-</option>
<option value="1">City 2</option>
<option value="2">City 1</option>
</select>
public void scrapeWebsite() {
WebClient webClient = new WebClient(BrowserVersion.CHROME);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setCssEnabled(false);
webClient.getOptions().setRedirectEnabled(true);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getCookieManager().setCookiesEnabled(true);
webClient.getOptions().setTimeout(90000);
String url = "www.xyz.com";
try {
HtmlPage page = webClient.getPage(url);
// drop down list 1
HtmlSelect districtNameSelect = (HtmlSelect) page.getElementById("ddldistlist");
HtmlOption districtNameOption = districtNameSelect.getOptionByValue("6");
districtNameSelect.setSelectedAttribute(districtNameOption,true);
String js = districtNameSelect.getOnChangeAttribute();
page = (HtmlPage) page.executeJavaScript(js).getNewPage();
webClient.waitForBackgroundJavaScript(5000);
// drop down list 2
HtmlSelect cityNameSelect = (HtmlSelect) page.getElementById("ddlaclist");
System.out.println("City name list : " + cityNameSelect.asText());
} catch (Exception exception) {
System.out.println("Exception:" + e.getMessage());
}
}
}