我正在尝试废弃此网站shopee.co.id
,但是当我将HTMlUnit page.asXml()
与浏览器检查进行比较时,结果却大不相同,HtmlUnit中的某些元素没有显示
来自HtmlUnit的结果:
...
<div id="main">
<!-- there is no element here -->
</div>
...
浏览器检查的结果:
...
<div id="main">
<!-- lot of element here-->
</div>
...
这是我的代码:
try (WebClient wc = new WebClient(BrowserVersion.FIREFOX_45)) {
wc.getOptions().setThrowExceptionOnFailingStatusCode(false);
wc.getOptions().setThrowExceptionOnScriptError(false);
WebRequest request = new WebRequest(new URL("https://www.shopee.co.id"));
HtmlPage page = wc.getPage(request);
System.out.println(page.asXml());
System.out.println("Done");
} catch (FailingHttpStatusCodeException | IOException e) {
System.out.println("Error while connecting");
}