这是我的代码:
from bs4 import BeautifulSoup as BS
import urllib2
url = "http://services.runescape.com/m=news/recruit-a-friend-for-free-membership-and-xp"
res = urllib2.urlopen(url)
soup = BS(res.read())
other_content = soup.find_all('div',{'class':'Content'})[0]
print other_content
然而出现了一个错误:
/Library/Python/2.7/site-packages/bs4/builder/_htmlparser.py:149: RuntimeWarning: Python's built-in HTMLParser cannot parse the given document. This is not a bug in Beautiful Soup. The best solution is to install an external parser (lxml or html5lib), and use Beautiful Soup with that parser. See http://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-a-parser for help.
"Python's built-in HTMLParser cannot parse the given document. This is not a bug in Beautiful Soup. The best solution is to install an external parser (lxml or html5lib), and use Beautiful Soup with that parser. See http://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-a-parser for help."))
Traceback (most recent call last):
File "web.py", line 5, in <module>
soup = BS(res.read())
File "/Library/Python/2.7/site-packages/bs4/__init__.py", line 172, in __init__
self._feed()
File "/Library/Python/2.7/site-packages/bs4/__init__.py", line 185, in _feed
self.builder.feed(self.markup)
File "/Library/Python/2.7/site-packages/bs4/builder/_htmlparser.py", line 150, in feed
raise e
我让其他两个人使用这个代码,它对他们非常好。为什么它不适合我?我安装了bs4 ......
答案 0 :(得分:6)
根据错误消息,您可能需要做的一件事就是安装lxml
,这将为BeautifulSoup提供更强大的解析引擎。有关更好的概述,请参阅文档中的this部分,但它可能适用于其他两个人的原因是他们安装了lxml
(或另一个正确处理HTML的解析器),这意味着BeautifulSoup使用它而不是标准的内置(侧面注释:您的示例在我安装lxml
的系统上也适用,但在没有它的情况下失败)。
另外,请参阅文档中的这条说明:
如果您使用的是早于2.7.3的Python 2版本或版本 在3.2.2之前的Python 3中,安装lxml至关重要 或html5lib-Python的内置HTML解析器不是很好 旧版本。
我建议运行sudo apt-get install python-lxml
并查看问题是否仍然存在。