为什么Beautifulsoup对此表什么也不返回？

Question

我正在尝试从Wunderground页面中提取特定表： https://www.wunderground.com/history/daily/us/ma/nantucket/KACK/date/2018-7-29

简而言之，该表称为“每日观察”。

从检查页面开始，表ID似乎是history-observation-table

我曾尝试使用BeautifulSoup，但是我想出的所有查找表（或任何表）的方法都行不通。

page = requests.get('https://www.wunderground.com/history/daily/us/ma/nantucket/KACK/date/2018-7-29').text


soup = bs(page.content,'html.parser')

soup.find_all("table")

结果为无/为空。我可以找到标题和div，但是如果要查找特定的类div则找不到。为什么我不能拉这张桌子？

Answer 1

页面正在使用javascript渲染表格，因此BeautifulSoup将不知道它在那里。您可以使用selenium获取正确的页面源并将其输入汤对象！

您将需要install selenium，此时脚本将变为：

from bs4 import BeautifulSoup as bs
from selenium import webdriver
import time

browser = webdriver.Chrome() # or some other browser
browser.get('https://www.wunderground.com/history/daily/us/ma/nantucket/KACK/date/2018-7-29')
time.sleep(2)
soup = bs(browser.page_source, 'html.parser')

print(soup.find_all("table"))

最好将time.sleep()替换为selenium waits

当我运行上面的脚本时，它会输出一个冗长的内容：

[<table _ngcontent-c14="" id="stationselector_table">
<tbody _ngcontent-c14="">
<!-- -->
</tbody>
</table>, <table _ngcontent-c7="">
<!-- --><!-- -->
<thead _ngcontent-c7="">
<tr _ngcontent-c7="">
<th _ngcontent-c7="">Temperature (° F)</th>
<!-- --><td _ngcontent-c7="">Actual</td><td _ngcontent-c7="">Historic Avg.</td><td _ngcontent-c7="">Record</td>
<td _ngcontent-c7="" style="height: 5px; width:10px;">
<svg _ngcontent-c7="" height="5" style="display: block" width="10" xmlns="http://www.w3.org/2000/svg"><polygon _ngcontent-c7="" fill="#000000" points="0,5 5,0 10,5"></polygon></svg>
</td>
</tr>
</thead>
<tbody _ngcontent-c7="">
<!-- --><tr _ngcontent-c7="">
<th _ngcontent-c7="">High Temp</th>
<!-- --><td _ngcontent-c7="">80</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">90</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Low Temp</th>
<!-- --><td _ngcontent-c7="">66</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">53</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Day Average Temp</th>
<!-- --><td _ngcontent-c7="">74</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr>
</tbody>
<!-- -->
<thead _ngcontent-c7="">
<tr _ngcontent-c7="">
<th _ngcontent-c7="">Precipitation (Inches)</th>
<!-- --><td _ngcontent-c7="">Actual</td><td _ngcontent-c7="">Historic Avg.</td><td _ngcontent-c7="">Record</td>
<td _ngcontent-c7="" style="height: 5px; width:10px;">
<svg _ngcontent-c7="" height="5" style="display: block" width="10" xmlns="http://www.w3.org/2000/svg"><polygon _ngcontent-c7="" fill="#000000" points="0,5 5,0 10,5"></polygon></svg>
</td>
</tr>
</thead>
<tbody _ngcontent-c7="">
<!-- --><tr _ngcontent-c7="">
<th _ngcontent-c7="">Precipitation</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">2.4</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Month to Date</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Year to Date</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr>
</tbody>
<!-- -->
<thead _ngcontent-c7="">
<tr _ngcontent-c7="">
<th _ngcontent-c7="">Degree Days (° F)</th>
<!-- --><td _ngcontent-c7="">Actual</td><td _ngcontent-c7="">Historic Avg.</td><td _ngcontent-c7="">Record</td>
<td _ngcontent-c7="" style="height: 5px; width:10px;">
<svg _ngcontent-c7="" height="5" style="display: block" width="10" xmlns="http://www.w3.org/2000/svg"><polygon _ngcontent-c7="" fill="#000000" points="0,5 5,0 10,5"></polygon></svg>
</td>
</tr>
</thead>
<tbody _ngcontent-c7="">
<!-- --><tr _ngcontent-c7="">
<th _ngcontent-c7="">Heating Degree Days</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">HDD Month to Date</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">HDD Since July 1</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Cooling Degree Days</th>
<!-- --><td _ngcontent-c7="">9</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">CDD Month to Date</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">CDD Year to Date</th>
<!-- --><td _ngcontent-c7="">0</td><td _ngcontent-c7="">0</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Growing Degree Days</th>
<!-- --><td _ngcontent-c7="">24</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr>
</tbody>
<!-- -->
<thead _ngcontent-c7="">
<tr _ngcontent-c7="">
<th _ngcontent-c7="">Dew Point (° F)</th>
<!-- --><td _ngcontent-c7="">Actual</td><td _ngcontent-c7="">Historic Avg.</td><td _ngcontent-c7="">Record</td>
<td _ngcontent-c7="" style="height: 5px; width:10px;">
<svg _ngcontent-c7="" height="5" style="display: block" width="10" xmlns="http://www.w3.org/2000/svg"><polygon _ngcontent-c7="" fill="#000000" points="0,5 5,0 10,5"></polygon></svg>
</td>
</tr>
</thead>
<tbody _ngcontent-c7="">
<!-- --><tr _ngcontent-c7="">
<th _ngcontent-c7="">Dew Point</th>
<!-- --><td _ngcontent-c7="">70</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">High</th>
<!-- --><td _ngcontent-c7="">72</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Low</th>
<!-- --><td _ngcontent-c7="">65</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Average</th>
<!-- --><td _ngcontent-c7="">70</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr>
</tbody>
<!-- -->
<thead _ngcontent-c7="">
<tr _ngcontent-c7="">
<th _ngcontent-c7="">Wind (MPH)</th>
<!-- --><td _ngcontent-c7="">Actual</td><td _ngcontent-c7="">Historic Avg.</td><td _ngcontent-c7="">Record</td>
<td _ngcontent-c7="" style="height: 5px; width:10px;">
<svg _ngcontent-c7="" height="5" style="display: block" width="10" xmlns="http://www.w3.org/2000/svg"><polygon _ngcontent-c7="" fill="#000000" points="0,5 5,0 10,5"></polygon></svg>
</td>
</tr>
</thead>
<tbody _ngcontent-c7="">
<!-- --><tr _ngcontent-c7="">
<th _ngcontent-c7="">Max Wind Speed</th>
<!-- --><td _ngcontent-c7="">9</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Visibility</th>
<!-- --><td _ngcontent-c7="">10</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr>
</tbody>
<!-- -->
<thead _ngcontent-c7="">
<tr _ngcontent-c7="">
<th _ngcontent-c7="">Sea Level Pressure (Hg)</th>
<!-- --><td _ngcontent-c7="">Actual</td><td _ngcontent-c7="">Historic Avg.</td><td _ngcontent-c7="">Record</td>
<td _ngcontent-c7="" style="height: 5px; width:10px;">
<svg _ngcontent-c7="" height="5" style="display: block" width="10" xmlns="http://www.w3.org/2000/svg"><polygon _ngcontent-c7="" fill="#000000" points="0,5 5,0 10,5"></polygon></svg>
</td>
</tr>
</thead>
<tbody _ngcontent-c7="">
<!-- --><tr _ngcontent-c7="">
<th _ngcontent-c7="">Sea Level Pressure</th>
<!-- --><td _ngcontent-c7="">30.11</td><td _ngcontent-c7="">-</td><td _ngcontent-c7="">-</td>
</tr>
</tbody>
<!-- -->
<thead _ngcontent-c7="">
<tr _ngcontent-c7="">
<th _ngcontent-c7="">Astronomy</th>
<!-- --><td _ngcontent-c7="">Day Length</td><td _ngcontent-c7="">Rise</td><td _ngcontent-c7="">Set</td>
<td _ngcontent-c7="" style="height: 5px; width:10px;">
<svg _ngcontent-c7="" height="5" style="display: block" width="10" xmlns="http://www.w3.org/2000/svg"><polygon _ngcontent-c7="" fill="#000000" points="0,5 5,0 10,5"></polygon></svg>
</td>
</tr>
</thead>
<tbody _ngcontent-c7="">
<!-- --><tr _ngcontent-c7="">
<th _ngcontent-c7="">Actual Time</th>
<!-- --><td _ngcontent-c7="">14h 28m</td><td _ngcontent-c7="">5:33 AM</td><td _ngcontent-c7="">8:02 PM</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Civil Twilight</th>
<!-- --><td _ngcontent-c7=""></td><td _ngcontent-c7="">5:02 AM</td><td _ngcontent-c7="">8:33 PM</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Nautical Twilight</th>
<!-- --><td _ngcontent-c7=""></td><td _ngcontent-c7="">4:23 AM</td><td _ngcontent-c7="">9:12 PM</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Astronomical Twilight</th>
<!-- --><td _ngcontent-c7=""></td><td _ngcontent-c7="">3:39 AM</td><td _ngcontent-c7="">9:56 PM</td>
</tr><tr _ngcontent-c7="">
<th _ngcontent-c7="">Moon: waning gibbous</th>
<!-- --><td _ngcontent-c7=""></td><td _ngcontent-c7="">9:13 PM</td><td _ngcontent-c7="">7:03 AM</td>
</tr>
</tbody>
</table>, <table _ngcontent-c17="" class="tablesaw-sortable" id="history-observation-table">
<thead _ngcontent-c17="">
<tr _ngcontent-c17="">
<!-- --><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-cell-persist tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Time</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-cell-persist tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Temperature</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Dew Point</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Humidity</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Wind</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Wind Speed</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Wind Gust</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Pressure</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Precip.</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Precip Accum</button>
</span></ngsaw-header>
</th><th _ngcontent-c17="">
<ngsaw-header _ngcontent-c17=""><span class="tablesaw-sortable-head">
<button class="tablesaw-sortable-btn">Condition</button>
</span></ngsaw-header>
</th>
</tr>
</thead>
<tbody _ngcontent-c17="">
<!-- -->
<!-- --><tr _ngcontent-c17="">
<!-- --><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>8:03 PM</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">68</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">68</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-humidity">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">100</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->%
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>SSW</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-speed">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">8</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->mph
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-speed">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->mph
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-pressure">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">29.9</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-rain">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0.0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-rain">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0.0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>Partly Cloudy</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td>
</tr><tr _ngcontent-c17="">
<!-- --><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>8:09 PM</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">69</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">69</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-humidity">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">100</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->%
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>SSW</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-speed">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">9</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->mph
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-speed">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->mph
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-pressure">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">29.9</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-rain">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0.0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-rain">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0.0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>Mostly Cloudy</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td>
</tr><tr _ngcontent-c17="">
<!-- --><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>8:51 PM</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">70</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">70</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-humidity">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">100</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->%
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>SW</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-speed">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">7</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->mph
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-speed">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->mph
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-pressure">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">29.9</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-rain">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0.0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-rain">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">0.0</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->in
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>Cloudy</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td>
</tr><tr _ngcontent-c17="">
<!-- --><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>8:53 PM</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">69</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test-true wu-unit wu-unit-temperature is-degree-visible">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">69</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->F
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-humidity">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">100</span> <span _ngcontent-c13="" class="wu-label">
<!-- -->%
    <!-- -->
</span>
<!-- -->
</span>
<!-- -->
</display-unit>
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- --><div>
<span>SW</span>
<!-- -->
</div>
<!-- -->
</ng-saw-cell-parser>
</td><td _ngcontent-c17="">
<ng-saw-cell-parser _ngcontent-c17=""><!-- -->
<!-- --><display-unit _nghost-c13=""><!-- --><span _ngcontent-c13="" class="test- wu-unit wu-unit-speed">
<!-- -->
<!-- -->
<!-- -->
<span _ngcontent-c13="" class="wu-value wu-value-to">7</span> <span _ngcontent-c13="" class="wu-label">
]

Process finished with exit code 0

实际上，这是一个很小的代码段，因为我的帖子限制为30,000个字符...

Answer 2

该网站正在使用angular js，您知道ng-s类是如何工作的，其次在sourceview上我找不到 table 标记，bs4也不如此

Answer 3

您使用了错误的链接，该代码以字典文件形式发送，您可以使用以下代码轻松访问该文件：

import requests

url = "https://api.weather.com:443/v1/geocode/41.28/-70.1/observations/historical.json?apiKey=6532d6454b8aa370768e63d6ba5a832e&startDate=20180729&endDate=20180729&units=e"
headers = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101 Firefox/66.0"}

all_observations = requests.get(url, headers=headers).json()["observations"] # The whole data in the table
first_observation = all_observations[0] # The first entry in the table

（PS：或者，您也可以按照其他答案的建议尝试使用无头浏览器）