Python 3请求或urllib - 如何始终添加标头?

时间:2016-09-13 19:43:11

标签: python python-requests urllib

标题说明了这一点:是否有一种“最佳”的方式来始终为每个请求添加标题?我有一个内部工具,想要将请求ID发送到其他内部工具;我正在寻找一个幸福的解决方案。我已经浏览了两者的文档,似乎这不是一个受欢迎的事情,因为我找不到食谱的例子。

我正在考虑一些解决方案:

  1. 在我自己的瘦包装器中包装请求并使用它。需要教导代码开发人员记住不是import requests而是import myrequestswrapper as requests
  2. Monkey-patch请求。我不喜欢猴子补丁,但也许这只是一次......?当我需要向这个特定系统发送标题时,我很害怕。
  3. 编辑:为什么我没有考虑request.Session:它存储cookie,需要处理,因为它保持连接打开。

1 个答案:

答案 0 :(得分:2)

创建session object,这是advanced usage下显示的第一件事:

s = requests.Session()
s.headers.update({'x-some-header': 'the value'})
s.get('http://httpbin.org/headers')

并使用会话执行请求。正如您所说,您不希望在请求之间保留cookie,您可以继承Session

In [64]: from requests.adapters import HTTPAdapter

In [65]: from requests.cookies import cookiejar_from_dict

In [66]: class CookieMonsterSession(Session):
    ...:     
    ...:     def __init__(self, *args, **kwgs):
    ...:         super(CookieMonsterSession, self).__init__(*args, **kwgs)
    ...:         # Override default adapters with 0-pooling adapters
    ...:         self.mount('https://', HTTPAdapter(pool_connections=1,
    ...:                                            pool_maxsize=0))
    ...:         self.mount('http://', HTTPAdapter(pool_connections=1,
    ...:                                           pool_maxsize=0))
    ...:     @property
    ...:     def cookies(self):
    ...:         """ Freshly baked cookies, always!"""
    ...:         return cookiejar_from_dict({})
    ...:     @cookies.setter
    ...:     def cookies(self, newcookies):
    ...:         """ OM NOM NOM NOM..."""
    ...:         pass
    ...:     

In [67]: s = CookieMonsterSession()

In [69]: real_s = Session()

In [70]: s.get('http://www.google.fi')
Out[70]: <Response [200]>

In [71]: s.cookies
Out[71]: <RequestsCookieJar[]>

In [72]: real_s.get('http://www.google.fi')
Out[72]: <Response [200]>

In [73]: real_s.cookies
Out[73]: <RequestsCookieJar[Cookie(version=0, name='NID', value='86=14qy...Rurx', port=None, port_specified=False, domain='.google.fi', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=1489744358, discard=False, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False)]>

遗憾的是Session设计难以扩展和配置,因此&#34;禁用&#34;如果Session更新,那么像这样的cookie是一种破解并容易中断。此外,我们仅针对持久标头禁用了Session的两个主要功能。

包装基本的API方法可能是更清洁,更安全的方法:

# customrequests.py
from functools import wraps
from requests import api as requests_api

custom_headers = {}


def _header_wrapper(f):
    @wraps(f)
    def wrapper(*args, **kwgs):
        headers = kwgs.pop('headers', None) or {}
        headers.update(custom_headers)
        return f(*args, headers=headers, **kwgs)

    return wrapper

request = _header_wrapper(requests_api.request)
get = _header_wrapper(requests_api.get)
options = _header_wrapper(requests_api.options)
head = _header_wrapper(requests_api.head)
post = _header_wrapper(requests_api.post)
put = _header_wrapper(requests_api.put)
patch = _header_wrapper(requests_api.patch)
delete = _header_wrapper(requests_api.delete)

行动中:

In [1]: import customrequests as requests

In [2]: print(requests.get('http://httpbin.org/headers').text)
{
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.11.1"
  }
}


In [3]: requests.custom_headers['X-Test'] = "I'm always here"

In [4]: print(requests.get('http://httpbin.org/headers').text)
{
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.11.1", 
    "X-Test": "I'm always here"
  }
}