Question

我正在学习如何使用html解析和操纵beautiful soup，如下所示：

from lxml.html import parse
import urllib2
from urllib2 import urlopen
from BeautifulSoup import BeautifulSoup

url = 'some-url-here'
req = urllib2.Request(url, headers={'User-Agent' : "Magic Browser"}) 
parsed = urllib2.urlopen( req )
soup = BeautifulSoup(parsed)

for elem in soup.findAll(['script', 'style', 'i']):
    elem.extract()

for main_body in soup.findAll("div", {"role" : "main"}):
    print main_body.getText(separator=u' ')

结果包含<i>个标签，我无法弄清楚如何删除它们。如何实现这一目标，为什么唯一的标签不能通过上述代码删除？

Answer 1

问题实际上是您使用已弃用的 Beautifulsoup3 ，安装bs4，一切都会正常工作：

In [10]: import urllib2
In [11]: from bs4 import BeautifulSoup # bs4

In [12]: url = 'https://www.gwr.com/'

In [13]: req = urllib2.Request(url, headers={'User-Agent': "Magic Browser"})

In [14]: parsed = urllib2.urlopen(req)

In [15]: soup = BeautifulSoup(parsed,"html.parser")

In [16]: tags = soup.find_all(['script','style','i'])

In [17]: print(len(tags))
25

In [18]: for elem in tags:
   ....:         elem.extract()
   ....:     

In [19]: assert len(soup.find_all(['script','style','i'])) == 0

In [20]:

BeautifulSoup不会删除i元素

1 个答案: