Question

我被Python中的requests代理所困扰。

下面是我的代码：

import requests
from lxml.html import fromstring
from bs4 import BeautifulSoup

url = 'https://free-proxy-list.net/'
response = requests.get(url)
parser = fromstring(response.text)
proxies = []
for i in parser.xpath('//tbody/tr')[:10]:
    if i.xpath('.//td[7][contains(text(),"yes")]'):
        #Grabbing IP and corresponding PORT
        proxy = ":".join([i.xpath('.//td[1]/text()')[0], i.xpath('.//td[2]/text()')[0]])
        proxies.append(proxy)

url = 'https://www.ebay.com'        
dic_proxies = {'https:':'http://' + str(proxies[0]), 
           'http':'http://' + str(proxies[0])}
u_a = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.82 Safari/537.36"
page = requests.get(url, proxies=dic_proxies,headers={"USER-AGENT":u_a})

soup = BeautifulSoup(page.text, 'html.parser')
p_soup = soup.prettify()

但是，上面的代码似乎只能从我的实际位置检索网站，而不能使用不同的代理访问世界其他地方的网站。我是否以正确的方式使用requests？还是我的代理无法正常工作？如何检查代理是否正常运行？

Answer 1

import requests
from requests.auth import HTTPBasicAuth

requests.get(url, auth=HTTPBasicAuth(<user>, <password>), stream=True, timeout=3)

如何使用请求代理以及如何检查代理是否与Python请求一起使用？

1 个答案: