urllib open - 如何控制重试次数

时间:2012-09-01 21:36:17

标签: python python-3.x urllib

如何控制opener.open的重试次数?

在下面的代码中,它会在引发异常或代码成功之前发送大约6个GET HTTP请求(我在Wireshark嗅探器中看到它)。

password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None,url, username, password)
handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
opener = urllib.request.build_opener(handler)

try: 
    resp = opener.open(url,None,1)

except urllib.error.URLError as e:
    print ("no success")
else:
    print ("success!")

1 个答案:

答案 0 :(得分:0)

urllib使用httplib来处理HTTP连接。如果您查看httplib.py来源并分析HTTPConnection课程,您会看到connect方法的工作原理。

def connect(self):
    """Connect to the host and port specified in __init__."""
    msg = "getaddrinfo returns an empty list"
    for res in socket.getaddrinfo(self.host, self.port, 0, socket.SOCK_STREAM):
        af, socktype, proto, canonname, sa = res
        try:
            self.sock = socket.socket(af, socktype, proto)
            if self.debuglevel > 0:
                print "connect: (%s, %s)" % (self.host, self.port)
            self.sock.connect(sa)
        except socket.error, msg:
            if self.debuglevel > 0:
                print 'connect fail:', (self.host, self.port)
            if self.sock:
                self.sock.close()
            self.sock = None
            continue
        break

    if not self.sock:
        raise socket.error, msg

实际上,尝试次数取决于socket.getaddrinfo返回的列表的长度。代码遍历所有可用的套接字选项以设置连接。

我怀疑减少尝试次数是有意义的。但你可以像下面这样增加这个数字:

maxtries = 3
delay = 3
n = 0
while n < maxtries:
    try: 
        resp = opener.open(url,None,1)
    except e:
        print ("no success")
        n += 1
        sleep(delay)
    else:
        print ("success!")
        break