使用美丽的汤解析html表

时间:2014-02-27 15:37:39

标签: python html parsing html-parsing beautifulsoup

我编写了这个用于打印表格的代码,如http://www.medindia.net/drug-price/list.asp

所示
import mechanize
import urllib2
from bs4 import BeautifulSoup

med="paracetamol"
br=mechanize.Browser()
br.set_handle_robots(False)
res=br.open("http://www.medindia.net/drug-price/")
br.select_form("frmdruginfo_search")
br.form['druginfosearch']=med
br.submit()
url=br.response().geturl()
print url
web_page = urllib2.urlopen(url)
soup = BeautifulSoup(web_page)
tabl=soup.find_all('table')
rows=tabl.find_all('tr')

for tr in rows:
        cols=tr.find_all('td')
        for td in cols:
              text = ''.join(td.find(text=True))
              print text+"|",

但是当我执行相同的操作时,我得到了这个错误

 rows=tabl.find_all('tr')
    AttributeError: 'list' object has no attribute 'find_all'

任何人都可以帮我解决这个问题吗?谢谢!

1 个答案:

答案 0 :(得分:3)

soup.find_all('table')会返回匹配表的列表,您只需要使用一个 - 使用find()

tabl = soup.find('table', {'class': 'content-table'})
rows = tabl.find_all('tr')

另请注意,我明确表示我需要一个具有特定类的表。

此外,您无需对该页面进行单独的urllib2调用 - 只需使用br.response().read()获取BS解析的实际html。

仅供参考,如果您想在控制台上获得更好的格式化表格结果,请考虑使用texttable

import mechanize
from bs4 import BeautifulSoup
import texttable


med = raw_input("Enter the drugname: ")
br = mechanize.Browser()
br.set_handle_robots(False)
res = br.open("http://www.medindia.net/drug-price/")
br.select_form("frmdruginfo_search")
br.form['druginfosearch'] = med
br.submit()

soup = BeautifulSoup(br.response().read())

tabl = soup.find('table', {'class': 'content-table'})
table = texttable.Texttable()
for tr in tabl.find_all('tr'):
    table.add_row([td.text.strip() for td in tr.find_all('td')])

print table.draw()

打印:

+--------------+--------------+--------------+--------------+--------------+
| SNo          | Prescribing  | Total No of  | Single       | Combination  |
|              | Information  | Brands       |     Generic  |     of       |
|              |              | (Single+Comb |              | Generic(s)   |
|              |              | ination)     |              |              |
+--------------+--------------+--------------+--------------+--------------+
| 1            | Abacavir     | 6            | View Price   | -            |
+--------------+--------------+--------------+--------------+--------------+
| 2            | Abciximab    | 1            | View Price   | -            |
+--------------+--------------+--------------+--------------+--------------+
| 3            | Acamprosate  | 3            | View Price   | -            |
+--------------+--------------+--------------+--------------+--------------+
| 4            | Acarbose     | 41           | View Price   | -            |
+--------------+--------------+--------------+--------------+--------------+
...