我编写了一个程序,用于在单击按钮后从网页中删除源代码。我无法抓住正确的页面,因为我相信正在发送一个AJAX请求,我不等待这个响应发生。我的代码目前是:
public class Htmlunitscraper {
private static String s = "http://cpdocket.cp.cuyahogacounty.us/SheriffSearch/results.aspx?q=searchType%3dSaleDate%26searchString%3d10%2f21%2f2013%26foreclosureType%3d%27NONT%27%2c+%27PAR%27%2c+%27COMM%27%2c+%27TXLN%27";
public static String scrapeWebsite() throws IOException {
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
final WebClient webClient = new WebClient();
final HtmlPage page = webClient.getPage(s);
final HtmlForm form = page.getForms().get(2);
final HtmlSubmitInput button = form.getInputByValue(">");
final HtmlPage page2 = button.click();
String originalHtml = page2.refresh().getWebResponse().getContentAsString();
return originalHtml;
}
}
在引用此link之后,我相信我可以解决此问题,我可以实现方法“webClient.waitForBackgroundJavaScript(10000)”。唯一的问题是我不明白该怎么做,因为每次单击按钮我都会创建一个HtmlPage对象,而不是WebClient对象。我怎样才能使用这种方法来解决问题?
答案 0 :(得分:7)
对我来说,它有助于将htmlunit 2.15与NicelyResynchronizingAjaxController一起使用,还有
webClient.getOptions().setThrowExceptionOnScriptError(false);
我的完整设置是
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setCssEnabled(false);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
答案 1 :(得分:3)
我会尝试设置
的解决方案webClient.setAjaxController(new NicelyResynchronizingAjaxController());
这会导致所有ajax调用同步。
或者,您是否尝试在解决方案中调用" webClient.waitForBackgroundJavaScript(10000)"在得到页面之后?
这样的事情:
final HtmlPage page2 = button.click();
webClient.waitForBackgroundJavaScript(10000)
String originalHtml = page2.asXml();
return originalHtml;
请同时使用htmlunit 2.13