从Web数据中提取两个表的内容

时间:2016-04-16 15:29:20

标签: python web-scraping beautifulsoup

如何从此html数据中检索所有td信息:

<h1>All staff</h1>
<h2>Manager</h2>
<table class="StaffList">
    <tbody>
        <tr>
            <th>Name</th>
            <th>Post title</th>
            <th>Telephone</th>
            <th>Email</th>
        </tr>
        <tr>
            <td>
                <a href="http://profiles.strx.usc.com/Profile.aspx?Id=Jon.Staut">Jon Staut</a>
            </td>
            <td>Line Manager</td>
            <td>0160 315 3832</td>
            <td>
                <a href="mailto:Jon.staut@strx.usc.com">Jon.staut@strx.usc.com</a> &nbsp;</td>
        </tr>
    </tbody>
</table>
<h2>Junior Staff</h2>
<table class="StaffList">
    <tbody>
        <tr>
            <th>Name</th>
            <th>Post title</th>
            <th>Telephone</th>
            <th>Email</th>
        </tr>
        <tr>
            <td>
                <a href="http://profiles.strx.usc.com/Profile.aspx?Id=Peter.Boone">Peter Boone</a>
            </td>
            <td>Mailer</td>
            <td>0160 315 3834</td>
            <td>
                <a href="mailto:Peter.Boone@strx.usc.com">Peter.Boone@strx.usc.com&nbsp;</a>
            </td>
        </tr>
        <tr>
            <td>
                <a href="http://profiles.strx.usc.com/Profile.aspx?Id=John.Peters">John Peters</a>
            </td>
            <td>Builder</td>
            <td>0160 315 3837</td>
            <td>
                <a href="mailto:John.Peters@strx.usc.com">John.Peters@strx.usc.com</a>
            </td>
        </tr>
    </tbody>
</table>

这是我生成错误的代码:

response =requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
table = soup.findAll('table', attrs={'class': 'StaffList'})

list_of_rows = []
for row in table.findAll('tr'): #2 rows found in table -loop through
    list_of_cells = []
    for cell in row.findAll('td'): # each cell in in a row
        text = cell.text.replace('&nbsp','')
        list_of_cells.append(text)
    #print list_of_cells
    list_of_rows.append(list_of_cells) 
#print all cells in the two rows
print list_of_rows 

错误讯息:

AttributeError: 'ResultSet' object has no attribute 'findAll'

如何使代码输出两个Web表中的所有信息,我需要做什么?

3 个答案:

答案 0 :(得分:2)

问题从这一行开始:

table = soup.findAll('table', attrs={'class': 'StaffList'})

findAll返回一个没有属性findAll的数组。

只需将findAll更改为find: table = soup.find(&#39; table&#39;,attrs = {&#39; class&#39;:&#39; StaffList&#39;})

答案 1 :(得分:1)

或者,您可以使用CSS选择器表达式从tr表返回 StaffList元素,而不必先提取table

for row in soup.select('table.StaffList tr'): #2 rows found in table -loop through
   ......

答案 2 :(得分:0)

感谢大家的建议。在替换2行代码后问题现在解决了:

第一个:

table = soup.findAll('table', attrs={'class': 'StaffList'})

替换为:

table = soup.findAll('tr')

第二个:

for row in table.findAll('tr'):

替换为:

for row in table: