Question

我使用htmlunit 2.36.0并尝试抓取： https://delightful.dussmann.com/menu/B%C3%BCropark%20Bredeney/B%C3%BCropark%20Bredeney 某种程度上，没有动态内容通过javascript执行。有人知道如何解决吗？

@Test
public void testPDFFetch() throws IOException {

    String url = "https://delightful.dussmann.com/menu/B%C3%BCropark%20Bredeney/B%C3%BCropark%20Bredeney";
    WebClient client = new WebClient(BrowserVersion.CHROME);
    client.getOptions().setJavaScriptEnabled(true);
    client.getOptions().setThrowExceptionOnScriptError(true);
    client.getOptions().setCssEnabled(true);
    client.getOptions().setUseInsecureSSL(true);

    client.setAjaxController(new AjaxController() {
        @Override
        public boolean processSynchron(HtmlPage page, WebRequest request, boolean async) {
            return true;
        }
    });
    try {
        HtmlPage page = client.getPage(url);
        // page.wait(20000);
        client.waitForBackgroundJavaScript(10000);
        client.waitForBackgroundJavaScriptStartingBefore(10000);
        Thread.sleep(10000);
        System.out.println(page.asXml());
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        client.close();
    }

}

Answer 1

此页面看起来使用了javascript模块-HtmlUnit（Rhino）当前不支持

使用HTMLUnit Web抓取Firebase页面

1 个答案: