从html页面中提取源代码

Question

我正在尝试使用以下代码从网站上阅读源代码

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;

public class GrabHTML {

public static void Connect() throws Exception{

 //Set URL
 URL url = new URL("http://www.google.ca/");
 URLConnection spoof = url.openConnection();

 //Spoof the connection so we look like a web browser
 spoof.setRequestProperty( "User-Agent", "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; H010818)" );
 BufferedReader in = new BufferedReader(new InputStreamReader(spoof.getInputStream()));
 String strLine = "";

 //Loop through every line in the source
   while ((strLine = in.readLine()) != null){

 //Prints each line to the console
   System.out.println(strLine);
  }

 System.out.println("End of page.");
}

public static void main(String[] args){

 try{
  //Calling the Connect method
  Connect();
 }catch(Exception e){

 }
 }
 }

但它只读取了部分源代码。当我从浏览器中看到“查看来源”时，Google.com会有更多数据。

Answer 1

删除以下声明

  

spoof.setRequestProperty（“User-Agent”，“Mozilla / 4.0（兼容; MSIE）   5.5; Windows NT 5.0; H010818）“）;