Question

我正在尝试使用Requests和BeautifulSoup从此页http://www.pro-football-reference.com/teams/nwe/2016_roster.htm上的games_played_team表中删除数据

url = "http://www.pro-football-
reference.com/teams/nwe/2016_roster.htm"
r = requests.get(url)

soup = BeautifulSoup(r.content)
print soup.prettify()

返回

<!DOCTYPE html>
<html data-version="klecko-" data-root="/home/pfr/build" 
itemscope="itemscope" itemtype="http://schema.org/WebSite" lang="en" class="no-js">
<head>
  <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=2.0" />
  <link rel="dns-prefetch" href="https://d2p3bygnnzw9w3.cloudfront.net/req/201706021" />
  <link href="https://d2p3bygnnzw9w3.cloudfront.net" rel="preconnect" crossorigin="crossorigin" />
  <link href="http://d9kjk42l7bfqz.cloudfront.net/req/201607120" rel="preconnect" crossorigin="crossorigin" />
  <link href="https://www.google-analytics.com/" rel="preconnect" crossorigin="crossorigin" />
  <link href="https://www.googletagservices.com" rel="preconnect" crossorigin="crossorigin" />
  <script class="allowed">
   var sr_is_production = true;
function vjs_ready(e){"loading"!=document.readyState?e():document.addEventListener("DOMContentLoaded",e)}var log_performance=!1,sr_detect_operaMini=navigator.userAgent.indexOf("Opera Mini")&gt
;-1;if(sr_detect_operaMini){var el=document.querySelector("html");el.className=el.className.concat(" operamini")}var sr_detect_firefox=navigator.userAgent.indexOf("Firefox")&gt;-1;if(sr_detect
_firefox){var el=document.querySelector("html");el.className=el.className.concat(" firefox")}var sr_detect_firefoxMobile=navigator.userAgent.indexOf("Firefox")&gt;-1&amp;&amp;(navigator.userAg
ent.indexOf("Mobile")&gt;-1||navigator.userAgent.indexOf("Tablet")&gt;-1);if(sr_detect_firefoxMobile){var el=document.querySelector("html");el.className=el.className.concat(" firefox-mobile")}
var sr_detect_ie=function(){var e=window.navigator.userAgent;if(e.indexOf("Trident/7.0")&gt;0)return 11;if(e.indexOf("Trident/6.0")&gt;0)return 10;if(e.indexOf("Trident/5.0")&gt;0)return 9;for
(var t=3,n=document.createElement("div"),r=n.getElementsByTagName("i");n.innerHTML="&lt;!--[if gt IE "+ ++t+"]&gt;&lt;i&gt;&lt;/i&gt;
&lt;link rel="shortcut icon" href="https://d2p3bygnnzw9w3.cloudfront.net/req/201706021/favicons/pfr/favicon.ico"&gt;
  </script>
 </head>
</html>

当我查看实际的页面源时，看起来我想要的表格是一个注释掉的HTML块。我不完全确定如何继续，并希望得到任何帮助

Answer 1

将requests.get更改为requests.post是最简单的方法。或者，如果您想要另一个简单的解决方案，您可以使用Selenium。

从注释掉的Python部分解析HTML

1 个答案: