Question

“urlib”库下是否有'requests'模块或'request'模块

运行以下代码时

from urllib.request import urlopen
from bs4 import BeautifulSoup
html = urlopen("http://pythonscraping.com/pages/page1.html")
bsObj = BeautifulSoup(html.read())
print (bsObj.h1)

shell正在抛出警告

Warning (from warnings module):
  File "C:\Python34\lib\site-packages\bs4\__init__.py", line 181
    markup_type=markup_type))
UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.
The code that caused this warning is on line 1 of the file <string>. To get rid of this warning, change code that looks like this:
 BeautifulSoup(YOUR_MARKUP})
to this:
 BeautifulSoup(YOUR_MARKUP, "html.parser")**

<h1>An Interesting Title</h1>

当我这样做时

>>>import requests

哪个shell成功导入请求模块

但是当我将上面的代码更改为

时

from urllib.requests import urlopen
from bs4 import BeautifulSoup
html = urlopen("http://pythonscraping.com/pages/page1.html")
bsObj = BeautifulSoup(html.read())
print (bsObj.h1)

shell抛出错误消息

Traceback (most recent call last):
  File "C:\Python34\scrapetest.py", line 1, in <module>
    from urllib.requests import urlopen
ImportError: No module named 'urllib.requests'

和pip工具成功安装模块（请求，请求）

C:Python34\Scripts>pip install request
Requirement already satisfied <use --upgrade to upgrade>: request in c:\python34\lib\site-packages
cleaning up...


C:Python34\Scripts>pip install requests
Requirement already satisfied <use --upgrade to upgrade>: requests in c:\python34\lib\site-packages
cleaning up...

Answer 1

test line 1test line 2是它拥有的模块，urllib也是如此。你这样做：

requests

它是import requests from urllib.request import urlopen而不是urllib.request，它解释了python解释器给你的错误。这是urllib.requests

的docs

至于你得到的第一个shell错误，python已经给你一个如何解决它的提示。所以，而不是：

urllib

你应该这样做：

bsObj = BeautifulSoup(html.read())

Answer 2

只需使用requests模块，这不是内置模块，而是第三方库。

import requests
from bs4 import BeautifulSoup

html = requests.get("http://pythonscraping.com/pages/page1.html").text
bsObj = BeautifulSoup(html, "html.parser")
print (bsObj.h1)

顺便说一句，警告与request(s)库无关，但它与BeautifulSoup解析器无关。

urllib.requests或urllib.request

2 个答案: