当我们遇到任何类型的错误(或超时)时,如何通过代理继续尝试urllib2请求?

时间:2016-06-19 11:08:14

标签: python urllib2

我正在使用urllib2方法(在for循环中)发出一个html请求,但有时我会得到错误或超时,所以我必须从for循环开始。任何人都可以告诉我如何修改我的代码,所以它一直在尝试,直到我们没有得到错误,而不是重新整理for循环?

Express server listening on port 3000
2016-06-19T09:37:39.331Z - error: Error occurred during table initialization ConnectionError: Login failed for user 'azuremobile'.  Reason: The password of the account must be changed.
    at Connection.<anonymous> (C:\Users\George\Source\Repos\backend\node_modules\mssql\lib\tedious.js:378:25)
    at Connection.g (events.js:260:16)
    at emitOne (events.js:77:13)
    at Connection.emit (events.js:169:7)
    at Connection.processLogin7Response (C:\Users\George\Source\Repos\backend\node_modules\tedious\lib\connection.js:672:16)
    at Connection.message (C:\Users\George\Source\Repos\backend\node_modules\tedious\lib\connection.js:1082:21)
    at Connection.dispatchEvent (C:\Users\George\Source\Repos\backend\node_modules\tedious\lib\connection.js:519:45)
    at MessageIO.<anonymous> (C:\Users\George\Source\Repos\backend\node_modules\tedious\lib\connection.js:439:23)
    at emitNone (events.js:67:13)
    at MessageIO.emit (events.js:166:7)

1 个答案:

答案 0 :(得分:-1)

正如我们在评论中所讨论的那样,您可以使用try/except来避免在循环播放时崩溃(我看到您在此建议之后更改了原始代码)

然后,您可以在使用urlopen时指定更长的超时时间(请参阅documentation)。

此外,在for循环中,您可以添加另一个循环,尝试检索数据一定次数,或在urlopen获得所需内容后立即中断。以下代码基于this answer

# number of attempt urlopen tries to open your url
attempts = 10

for _ in range(attempts):
    try:
        # you can use timeout argument for urlopen
        html = urllib2.urlopen(req, timeout=timeout).read()
        # urlopen successfully get the data
        break
    # urlopen fails to retrieve data
    except urllib2.URLError as err:
        print("Oops! urlopen failed")
# all attempts failed
else:
    print("Oops! All attempts failed")
# now use your html variable here
# ...

P.S。对于那些被投票的人:OP在评论中讨论后改变了他的问题/代码。这个答案是该讨论的后续行动,因此请考虑上下文。