我想在我的代码段中实现代理。我怎样才能做到这一点?
try:
raw = self.session.get(self.BASE_URL + '/archive').text
except:
logging.info('Error with pastebin')
raw = None
sleep(5)
results = BeautifulSoup(raw).find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])
答案 0 :(得分:1)
您可以导入urllib并尝试执行以下操作来实现代理
try:
raw = self.session.get(self.BASE_URL + '/archive').text
except:
proxy_support = urllib2.ProxyHandler({"http":"http://10.62.51.201:3128"})
opener = urllib2.build_opener(proxy_support)
urllib2.install_opener(opener)
#results = BeautifulSoup(helper.download(urllib2.urlopen(self.BASE_URL).geturl())+ '/archive').find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])
myurl = urllib2.urlopen(self.BASE_URL).geturl()
raw = self.session.get(myurl + '/archive').text
results = BeautifulSoup(raw).find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])