使用urllib2从基于身份验证的Jenkins服务器获取URL

Question

我正在尝试从Jekins服务器获取URL。直到最近，我才能够使用此页面上描述的模式（HOWTO Fetch Internet Resources Using urllib2）来创建一个密码管理器，该密码管理器使用用户名＆amp ;;正确响应BasicAuth挑战。密码。一切都很好，直到Jenkins团队changed their security model，并且该代码不再有效。

# DOES NOT WORK!
import urllib2
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
top_level_url = "http://localhost:8080"

password_mgr.add_password(None, top_level_url, 'sal', 'foobar')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(handler)

a_url = 'http://localhost:8080/job/foo/4/api/python'
print opener.open(a_url).read()

堆栈跟踪：

Traceback (most recent call last):
  File "/home/sal/workspace/jenkinsapi/src/examples/password.py", line 11, in <module>
    print opener.open(a_url).read()
  File "/usr/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden
[Finished in 0.0s with exit code 1]

问题似乎是Jenkins没有返回预期的401代码，而是一个403，urllib2将其解释为会话结束。它实际上从未发送过密码。在github周围进行一些冲浪之后，找到了另一个开发人员的解决方案......

# WORKS... SORTA
def auth_headers(username, password):
   return 'Basic ' + base64.encodestring('%s:%s' % (username, password))[:-1]

auth = auth_headers('sal', 'foobar')
top_level_url = "http://localhost:8080"
a_url = 'http://localhost:8080/job/foo/4/api/python'
req = urllib2.Request(a_url)
req.add_header('Authorization', auth)
print urllib2.urlopen(req).read()

但这似乎并不令人满意。检查域名是否与用户名和密码相关并不困难...它只是发送我的登录详细信息而不管！

有人可以建议一种让原始脚本有效的方法吗？我想以一种我可以登录Jenkins的方式使用urllib2密码管理器。

Answer 1

也可以看到这个要点：https://gist.github.com/dnozay/194d816aa6517dc67ca1

当您需要访问需要身份验证的页面时，Jenkins不会返回401 - retry HTTP错误代码;相反，它返回403 - forbidden。在wiki https://wiki.jenkins-ci.org/display/JENKINS/Authenticating+scripted+clients中，它显示使用命令行工具wget，您需要使用wget --auth-no-challenge，这正是因为这种行为。

获得403 - forbidden时重试基本身份验证：

让我们说你定义了：

jenkins_url = "https://jenkins.example.com"
username = "johndoe@example.com"
api_token = "my-api-token"

您可以将urllib2.HTTPBasicAuthHandler子类化为处理403 HTTP响应。

import urllib2

class HTTPBasic403AuthHandler(urllib2.HTTPBasicAuthHandler):
    # retry with basic auth when facing a 403 forbidden
    def http_error_403(self, req, fp, code, msg, headers):
        host = req.get_host()
        realm = None
        return self.retry_http_basic_auth(host, req, realm)

然后是使用该处理程序的问题，例如您可以安装它，以便适用于所有urllib2.urlopen次呼叫：

def install_auth_opener():
    '''install the authentication handler.

    This handles non-standard behavior where the server responds with
    403 forbidden, instead of 401 retry. Which means it does not give you the
    chance to provide your credentials.'''
    auth_handler = HTTPBasic403AuthHandler()
    auth_handler.add_password(
        realm=None,
        uri=jenkins_url,
        user=username,
        passwd=api_token)
    opener = urllib2.build_opener(auth_handler)
    # install it for all urllib2.urlopen calls
    urllib2.install_opener(opener)

这是一个简单的测试，看看它是否正常。

if __name__ == "__main__":
    # test
    install_auth_opener()
    page = "%s/me/api/python" % jenkins_url
    try:
        result = urllib2.urlopen(page)
        assert result.code == 200
        print "ok"
    except urllib2.HTTPError, err:
        assert err.code != 401, 'BAD CREDENTIALS!'
        raise err

使用先发制人认证。

这个答案中有一个很好的例子：https://stackoverflow.com/a/8513913/1733117。 当您获得403 forbidden时，您可以在网址匹配时发送Authorization标头，而不是重试。

class PreemptiveBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
    '''Preemptive basic auth.

    Instead of waiting for a 403 to then retry with the credentials,
    send the credentials if the url is handled by the password manager.
    Note: please use realm=None when calling add_password.'''
    def http_request(self, req):
        url = req.get_full_url()
        realm = None
        # this is very similar to the code from retry_http_basic_auth()
        # but returns a request object.
        user, pw = self.passwd.find_user_password(realm, url)
        if pw:
            raw = "%s:%s" % (user, pw)
            auth = 'Basic %s' % base64.b64encode(raw).strip()
            req.add_unredirected_header(self.auth_header, auth)
        return req

    https_request = http_request

Answer 2

不是定义自己的处理程序并将其全局安装或将其用于单个请求，而是将标头添加到请求中要容易得多：

auth_header = 'Basic ' + base64.b64encode('%s:%s' % (USERNAME,
                                                      API_KEY)).strip()
headers = {'Authorization': auth_header}

request = urllib2.Request(url, urllib.urlencode(data), headers)
result = urllib2.urlopen(request)