以下是一些数据:
<tr>
<td data-th='Stock Symbol'>
<a href="/dividend-stocks/technology/personal-computers/aapl-apple-inc/">AAPL</a>
</td>
<td data-th='Company Name'>
<a href="/dividend-stocks/technology/personal-computers/aapl-apple-inc/">Apple Inc.<div class='label label-primary' style='margin-left: 5px;'>
</div></a>
</td>
<td data-th='DARS™ Rating'>
<a href='/premium/signup.php' class='restricted' style='vertical-align: middle;'></a>
</td>
<td data-th='Dividend Yield'>
1.65%
</td>
<td data-th='Closing Price'>
$153.18
</td>
<td data-th='Annualized Dividend'>
$2.52
</td>
<td data-th='Ex-Div Date'>
<span style="white-space: nowrap;">2017-05-11</span>
</td>
<td data-th='Pay Date'>
<span style="white-space: nowrap;">2017-05-18</span>
</td>
</tr>
我需要得到1.65%,$ 153.18和$ 2.52的值。它们都是一行没有标签。
import requests
from bs4 import BeautifulSoup
url = "http://www.dividend.com/dividend-stocks/dow-30-dividend-stocks.php"
r = requests.get(url)
soup = BeautifulSoup(r.content, "html.parser")
for tds in soup.find_all("td"):
print(tds)
答案 0 :(得分:1)
我发现html.parser
不是这种情况下的最佳选择。让我们试试html5lib
。输入(linux)
sudo apt-get install python-html5lib
安装新的解析器。 Link to BF+html5lib docs
这是工作代码(用于打印上述tds的文本):
import requests
from bs4 import BeautifulSoup
url = "http://www.dividend.com/dividend-stocks/dow-30-dividend-stocks.php"
r = requests.get(url)
soup = BeautifulSoup(r.content, "html5lib")
interesting_tds = ['Dividend Yield', 'Closing Price', 'Annualized Dividend']
for td in soup.find_all("td"):
if td.get('data-th') in interesting_tds:
print(td.text.strip())
# or just process td object