我具有此功能,当达到速率限制时会再次调用自身。它最终应该成功并返回工作数据。它正常工作,然后速率限制按预期工作,最后,当数据恢复正常时,我得到:
TypeError:“ NoneType”对象不可下标
def grabPks(pageNum):
# cloudflare blocks bots...use scraper library to get around this or build your own logic to store and use a manually generated cloudflare session cookie... I don't care
req = scraper.get("sumurl.com/"+str(pageNum)).content
if(req == b'Rate Limit Exceeded'):
print("adjust the rate limiting because they're blocking us :(")
manPenalty = napLength * 3
print("manually sleeping for {} seconds".format(manPenalty))
time.sleep(manPenalty)
print("okay let's try again... NOW SERVING {}".format(pageNum))
grabPks(pageNum)
else:
tree = html.fromstring(req)
pk = tree.xpath("/path/small/text()")
resCmpress = tree.xpath("path/a//text()")
resXtend = tree.xpath("[path/td[2]/small/a//text()")
balance = tree.xpath("path/font//text()")
return pk, resCmpress, resXtend, balance
我试图将返回值移到else范围之外,但随后抛出:
UnboundLocalError:分配前已引用本地变量“ pk”
答案 0 :(得分:0)
好吧...我需要返回grabPK才能使其正常运行...:
.jpg
答案 1 :(得分:0)
如果您的顶级grabPks
受速率限制,则不会返回任何内容。
考虑一下:
grabPks()
grabPks()
。grabPks()
将值返回到它上面的函数。在您的return grabPks(pageNum)
块中尝试使用if
。