尝试使用jSoup解析网页时遇到问题。如果我在Chrome中查看页面源代码，则会显示您期望的所有内容 - 整页。但是，当我使用JSoup连接到url时，Jsoup Document只包含少量页面的html / xml。
知道为什么会这样吗？
以下是差异的示例：
页面来源：
<title>Bath Rugby Official Website | First team</title>
<base href="http://www.bathrugby.com/"></base>
<meta name="description" content="The Official Website of Bath Rugby. The only place for in-depth player, match and club news." /><meta property="og:title" content="Bath Rugby Official Website | First team" /><meta property="og:url" content="http://www.bathrugby.com/fixtures-and-results/first-team/" />
<meta property="og:type" content="sports_team" />
<!-- <meta property="og:image" content="http://www.bathrugby.com/_assets/img/common/bath_rugby_logo.jpg" /> -->
<meta property="og:site_name" content="Bath Rugby" />
<meta property="fb:admins" content="100001941260923" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Latest News - RSS feed" href="/rss/rss-all-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Match News - RSS feed" href="/rss/rss-match-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Team News - RSS feed" href="/rss/rss-team-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Club News - RSS feed" href="/rss/rss-club-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Academy News - RSS feed" href="/rss/rss-academy-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Ladies News - RSS feed" href="/rss/rss-ladies-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Community News - RSS feed" href="/rss/rss-community-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Programmes and Projects - Inclusion - RSS feed" href="/rss/rss-programmes-and-projects-inclusion.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Programmes and Projects - Education - RSS feed" href="/rss/rss-programmes-and-projects-education.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Programmes and Projects - Sports - RSS feed" href="/rss/rss-programmes-and-projects-sport.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Corporate News - RSS feed" href="/rss/rss-corporate-news.rss" />
<link rel="alternate" type="application/rss+xml" title="Bath Rugby - Redevelopment News - RSS feed" href="/rss/rss-redevelopment-news.rss" />
JSoup Document：
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Bath Rugby - Fixtures and results</title>
<meta name="Author" content="Positive -> http://www.positivestudio.co.uk" />
<meta name="HandheldFriendly" content="True" />
<meta name="viewport" content="width=device-width; initial-scale=1.0; minimum-scale=1.0; maximum-scale=1.0; user-scalable=0;" />
<meta name="google-site-verification" content="7dRcSqMI5R3Rag9iNiHeFei6ycyBTd3dmhlZvF8QdhA" />
<link rel="Shortcut Icon" type="image/ico" href="/_assets/favicon.ico" />
<link rel="apple-touch-icon-precomposed" href="/_assets/img/icons/apple-touch-icon.png" />
<link rel="stylesheet" type="text/css" media="all" href="/_assets/css/mobile.css" />
<link rel="stylesheet" type="text/css" href="/_assets/css/retina.css" media="only screen and (-webkit-min-device-pixel-ratio: 2)" />
<script type="text/javascript" src="/_assets/js/jquery-1.4.2.min.js"></script>
<script src="/_assets/js/css_browser_selector.js" type="text/javascript"></script>
<script type="text/javascript" src="/_assets/js/scripts.js"></script>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-24788688-3']);
_gaq.push(['_trackPageview']);
(function() {
答案 0 :(得分：0)
网站似乎检测到您正在浏览Android并提供内容较少的适合移动设备的网页。比较您在Chrome和Android浏览器中查看时的http://www.bathrugby.com内容。
您可以通过执行类似connection.userAgent(params, "Mozilla/4.0")的操作来影响通过Jsoup提供的服务。
答案 1 :(得分：0)
我通过将页面的源html检索为字符串，然后将其传递给jSoup来回答这个问题。
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
String html = "";
InputStream in = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder str = new StringBuilder();
String line = null;
while((line = reader.readLine()) != null)
{
str.append(line);
}
in.close();
html = str.toString();
doc = Jsoup.parse(html);