使用urllib,urllib2和request避免使用“Pyram of Doom”语法

时间:2017-12-21 01:09:22

标签: python request urllib try-except

很难为响应编写Python2和Python3以及request依赖项代码,因为它们urlopen()函数和requests.get()函数返回不同的类型:

  • Python2 urllib.request.urlopen()返回http.client.HTTPResponse
  • Python3 urllib.urlopen(url)返回instance
  • 请求request.get(url)返回requests.models.Response

为了支持Python2和Python3以及不想安装request依赖关系的用户,我尝试了导入中try-except的{​​{1}}的“金字塔末日”和{{1功能:

get_content()

还有其他方法可以获得返回阅读内容并避免嵌套try: # Try importing requests first. import requests except ImportError: try: # Try importing Python3 urllib import urllib.request except AttributeError: # Now importing Python2 urllib import urllib def get_content(url): try: # Using requests. return requests.get(url).content # Returns requests.models.Response. except NameError: try: # Using Python3 urllib. with urllib.request.urlopen(index_url) as response: return response.read() # Returns http.client.HTTPResponse. except AttributeError: # Using Python3 urllib. return urllib.urlopen(url).read() # Returns an instance. 的相同结果吗?

3 个答案:

答案 0 :(得分:7)

您的用户使用requests有什么好处?简单地忽略requests并支持标准库函数更容易。通过这样导入,可以透明地完成其余的代码:

try:
    from urllib.request import urlopen
except ImportError:
    from urllib2 import urlopen

从那时起,所有GET请求都可以urlopen(url)发出。可以使用read()检索返回的数据。

现在,如果您真的想继续requests支持,您可以像这样编写导入代码以及get_content()的定义:

try:
    import requests
    get_content = lambda url : requests.get(url).content
except ImportError:
    try:                   # Python 3
        from urllib.request import urlopen
    except ImportError:    # Python2
        from urllib2 import urlopen
    get_content = lambda url : urlopen(url).read()

答案 1 :(得分:1)

正如大家所提到的,如果你要去urllib的路线,你不应该过多地讨论请求。但是,如果你仍然希望摆脱太多嵌套异常,你可以使用类似下面的内容

import sys
key = str(sys.version_info.major)

try:  # Try importing requests first.
    import requests
    key = "requests"
except ImportError:
    try:  # Try importing Python3 urllib
        import urllib.request
    except (AttributeError, ImportError):  # Now importing Python2 urllib
        import urllib

FUNC = {
    "2": {
        "urlopen": lambda url: urllib.urlopen(url),
        "read": lambda res: res.read()
    },
    "requests": {
        "urlopen": lambda url: requests.get(url),
        "read": lambda res: res.content
    },
    "3": {
        "urlopen": lambda url: urllib.request.urlopen(url),
        "read": lambda res: res.read()
    },
}

urlopen = FUNC[key]["urlopen"]
read = FUNC[key]["read"]

def get_content(url):
    res = urlopen(url)
    return read(res)

print(get_content("http://tarunlalwani.com"))

答案 2 :(得分:0)

您可以尝试将所有三个库导入为request,并检查get_content函数中的属性以识别导入的库。
作为嵌套的try-except块的替代方法,您可以在单独的try-except块中导入requests, 如果安装了模块,它将覆盖urllib / urllib2

try: 
    import urllib.request as request
except ImportError: 
    import urllib2 as request
try: 
    import requests as request
except ImportError:
    pass

def get_content(url):
    return request.get(url).content if hasattr(request, 'get') else request.urlopen(url).read()

这会产生干净和sipmle代码,并允许您汇总更复杂的请求,例如,

def get_content(url, headers={}):
    if hasattr(request, 'get'):
        return request.get(url, headers=headers).content
    else:
        return request.urlopen(request.Request(url, headers=headers)).read()