Question

我试图获取在加载所有javascript后生成的div的内容，但它并不重要我做什么，它似乎忽略了javascript并只返回其中的页面＆＃39 ; s＆＃34;版本＆＃34;。

我已经在stackoverflow中搜索了有关此问题的所有问题，但任何答案都已解决..

这是我的代码：

    WebClient webClient = null;

    HtmlPage page = null;

    webClient = new WebClient(BrowserVersion.CHROME);

    webClient.getOptions().setThrowExceptionOnScriptError(false);

    webClient.getOptions().setJavaScriptEnabled(true);

    try {
        WebRequest request = new WebRequest(new URL("myUrl"));

        page = webClient.getPage(request);

        webClient.setAjaxController(new NicelyResynchronizingAjaxController());

        List<DomElement> divs = page.getElementsByTagName("div");   

        while(divs.size() <= 46){
            synchronized (page) {
                page.wait(2000); //wait
                    webClient.waitForBackgroundJavaScriptStartingBefore(10000);
                    webClient.waitForBackgroundJavaScript(10000);
                }
            divs = page.getElementsByTagName("div");
        }

        String content = page.getWebResponse().getContentAsString();                

    } catch (Exception e) {
        e.printStackTrace();
    }

Answer 1

对于我不久前遇到的类似情况，我发现只需将当前线程置于休眠状态几秒钟就可以使远程JavaScript完成运行。以下是我配置Web客户端的方法：

private WebClient getConfiguredWebClient () {
    WebClient aClient = null;
    switch (Configuration.getString("AdMonitor.browser")) {
        case "chrome" :
            aClient = new WebClient(BrowserVersion.CHROME);
            break;
        case "firefox" :
            aClient = new WebClient(BrowserVersion.FIREFOX_52);
            break;
        case "ie" :
            aClient = new WebClient(BrowserVersion.INTERNET_EXPLORER);
            break;
        case "edge" :
            aClient = new WebClient(BrowserVersion.EDGE);
            break;
        default :
            LOGGER.warn("Browser specified in configuration file (" + Configuration.getString("Monitor.browser") + ") is unsupported - defaulting to \"BEST_SUPPORTED\" . . .");
            aClient = new WebClient(BrowserVersion.BEST_SUPPORTED);
    }
    aClient.waitForBackgroundJavaScript(3 * 1000); // Experimental API: May be changed in next release and may not yet work perfectly!
    aClient.getOptions().setCssEnabled(true);
    aClient.getOptions().setJavaScriptEnabled(true);
    aClient.setAjaxController(new NicelyResynchronizingAjaxController());
    LOGGER.info(new StringBuilder(64).append("Browser in use: ").append(aClient.getBrowserVersion()));
    return aClient;
}

这就是我如何得到有问题的页面来完成加载：

private HtmlPage getNextPage(WebClient pClient) throws IOException {
    HtmlPage apagLanding = pClient.getPage(new java.net.URL(LANDING_PAGE_URL));
    HtmlAnchor anchor = (HtmlAnchor) apagLanding.getElementById("all");
    HtmlPage apagAllAds = anchor.click();
    try {
        LOGGER.info("Pausing for JavaScript execution to return page . . .");
        Thread.sleep(1000 * 2);
    } catch (InterruptedException e) {
        LOGGER.info("InterruptedException encountered (non-critical condition) . . .", e);
    }
    return apagAllAds;
}

Answer 2

如果您愿意，可以查看我用于Wetator的代码。检查HtmlUnitBrowser.java;从方法waitForImmediateJobs（）或assertContentInTimeFrame（）开始。

HTMLUnit不等待javascript

2 个答案: