BeautifulSoup:从Children节点中提取值

时间:2013-01-27 02:12:28

标签: python python-2.7 beautifulsoup

我有以下html:

<td class="section">
    <div style="margin-top:2px; margin-bottom:-10px; ">
    <span class="username"><a href="user.php?id=xx">xxUsername</a></span>
    </div>
    <br>
<span class="comment">
A test comment
</span>
</td>

我想要在SPAN标记内检索xxUsername和注释文本。到目前为止,我已经这样做了:

results = soup.findAll("td", {"class" : "section"})

它确实获取了我上面提到的模式的所有html块。现在我想在一个循环中检索所有子值?可能吗?如果没有,那么如何获取子节点信息?

2 个答案:

答案 0 :(得分:7)

你可以尝试这样的事情。它基本上完成了你上面所做的事情 - 首先遍历所有section - 分类td,然后遍历所有span文本。这打印出了类,以防你需要更严格的限制:

In [1]: from bs4 import BeautifulSoup

In [2]: html = # Your html here

In [3]: soup = BeautifulSoup(html)

In [4]: for td in soup.find_all('td', {'class': 'section'}):
   ...:     for span in td.find_all('span'):
   ...:         print span.attrs['class'], span.text
   ...:         
['username'] xxUsername
['comment'] 
A test comment

或者使用更加复杂的单行程,将所有内容存储在您的列表中:

In [5]: results = [span.text for td in soup.find_all('td', {'class': 'section'}) for span in td.find_all('span')]

In [6]: results
Out[6]: [u'xxUsername', u'\nA test comment\n']

或者在同一个主题上,一个字典的键是类的元组,值是文本本身:

In [8]: results = dict((tuple(span.attrs['class']), span.text) for td in soup.find_all('td', {'class': 'section'}) for span in td.find_all('span'))

In [9]: results
Out[9]: {('comment',): u'\nA test comment\n', ('username',): u'xxUsername'}

假设这个更接近你想要的,我建议重写为:

In [10]: results = {}

In [11]: for td in soup.find_all('td', {'class': 'section'}):
   ....:     for span in td.find_all('span'):
   ....:         results[tuple(span.attrs['class'])] = span.text
   ....:         

In [12]: results
Out[12]: {('comment',): u'\nA test comment\n', ('username',): u'xxUsername'}

答案 1 :(得分:1)

usernamecomment <span>元素中获取文字:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html)
for el in soup('span', ['username', 'comment']):
    print el.string,

输出

xxUsername 
A test comment