如何在beautifulsoup中处理没有标签的td数据?

时间:2017-06-02 19:47:45

标签: python-3.x beautifulsoup

以下是一些数据:

<tr>
<td data-th='Stock Symbol'>
<a href="/dividend-stocks/technology/personal-computers/aapl-apple-inc/">AAPL</a>
</td>
<td data-th='Company Name'>
<a href="/dividend-stocks/technology/personal-computers/aapl-apple-inc/">Apple Inc.<div class='label label-primary' style='margin-left: 5px;'>
</div></a>
</td>
<td data-th='DARS™ Rating'>
<a href='/premium/signup.php' class='restricted' style='vertical-align: middle;'></a>
</td>
<td data-th='Dividend Yield'>
1.65%
</td>
<td data-th='Closing Price'>
$153.18
</td>
<td data-th='Annualized Dividend'>
$2.52
</td>
<td data-th='Ex-Div Date'>
<span style="white-space: nowrap;">2017-05-11</span>
</td>
<td data-th='Pay Date'>
<span style="white-space: nowrap;">2017-05-18</span>
</td>
</tr>

我需要得到1.65%,$ 153.18和$ 2.52的值。它们都是一行没有标签。

此代码不返回任何内容。我怎么能绕过这个?感谢。

import requests
from bs4 import BeautifulSoup
url = "http://www.dividend.com/dividend-stocks/dow-30-dividend-stocks.php"
r = requests.get(url)
soup = BeautifulSoup(r.content, "html.parser")

for tds in soup.find_all("td"):
   print(tds)

1 个答案:

答案 0 :(得分:1)

我发现html.parser不是这种情况下的最佳选择。让我们试试html5lib。输入(linux)

sudo apt-get install python-html5lib

安装新的解析器。 Link to BF+html5lib docs

这是工作代码(用于打印上述tds的文本):

import requests
from bs4 import BeautifulSoup
url = "http://www.dividend.com/dividend-stocks/dow-30-dividend-stocks.php"
r = requests.get(url)
soup = BeautifulSoup(r.content, "html5lib")
interesting_tds = ['Dividend Yield', 'Closing Price', 'Annualized Dividend']

for td in soup.find_all("td"):
    if td.get('data-th') in interesting_tds:
        print(td.text.strip())
        # or just process td object