我写了一个抓取工具来从Q& A网站上获取信息。由于并非所有字段都始终显示在页面中,因此我使用了多个try-excepts来处理这种情况。
def answerContentExtractor( loginSession, questionLinkQueue , answerContentList) :
while True:
URL = questionLinkQueue.get()
try:
response = loginSession.get(URL,timeout = MAX_WAIT_TIME)
raw_data = response.text
#These fields must exist, or something went wrong...
questionId = re.findall(REGEX,raw_data)[0]
answerId = re.findall(REGEX,raw_data)[0]
title = re.findall(REGEX,raw_data)[0]
except requests.exceptions.Timeout ,IndexError:
print >> sys.stderr, URL + " extraction error..."
questionLinkQueue.task_done()
continue
try:
questionInfo = re.findall(REGEX,raw_data)[0]
except IndexError:
questionInfo = ""
try:
answerContent = re.findall(REGEX,raw_data)[0]
except IndexError:
answerContent = ""
result = {
'questionId' : questionId,
'answerId' : answerId,
'title' : title,
'questionInfo' : questionInfo,
'answerContent': answerContent
}
answerContentList.append(result)
questionLinkQueue.task_done()
此代码有时可能会或可能不会在运行时发出以下异常:
UnboundLocalError: local variable 'IndexError' referenced before assignment
行号表示第二个except IndexError:
感谢大家的建议,我很乐意给你应得的印记,太糟糕了我只能将其标记为正确答案......
答案 0 :(得分:6)
我认为问题在于这一行:
except requests.exceptions.Timeout ,IndexError
这相当于:
except requests.exceptions.Timeout as IndexError:
因此,您要将IndexError
分配给requests.exceptions.Timeout
捕获的异常。错误可以通过以下代码重现:
try:
true
except NameError, IndexError:
print IndexError
#name 'true' is not defined
要捕获多个异常,请使用元组:
except (requests.exceptions.Timeout, IndexError):
UnboundLocalError
即将到来,因为您的函数将IndexError
视为局部变量,因此在实际定义之前尝试访问其值会引发UnboundLocalError
错误。
>>> 'IndexError' in answerContentExtractor.func_code.co_varnames
True
因此,如果此行未在运行时执行(requests.exceptions.Timeout ,IndexError
),则其下方使用的IndexError
变量将引发UnboundLocalError
。重现错误的示例代码:
def func():
try:
print
except NameError, IndexError:
pass
try:
[][1]
except IndexError:
pass
func()
#UnboundLocalError: local variable 'IndexError' referenced before assignment
答案 1 :(得分:2)
当你说
时except requests.exceptions.Timeout ,IndexError:
Python除requests.exceptions.Timeout
错误外,错误对象为IndexError
。应该是这样的
except (requests.exceptions.Timeout ,IndexError) as e:
答案 2 :(得分:1)
except requests.exceptions.Timeout ,IndexError:
与except requests.exceptions.Timeout as IndexError
你应该使用
except (requests.exceptions.Timeout, IndexError):
代替