如何在请求中更改用户代理或VPN?

时间:2019-04-23 20:29:58

标签: python ajax web-scraping request

我的代码有问题。我正在尝试抓取页面,但是它发送给我一个“被阻止”的错误,我认为我已经被禁止了...。我想知道如何更改用户代理或VPN来使该页面不被禁止。

   lugarDeseado="Egipto"
   paises=['ac',
   'ad',
   'ae',
   'af',
   'ag',
   'ai',
   'al',
   'am',
   'an',
   'ao',
   'aq',
   'ar',
   'es',
   'vn',
   'vu',
   'wf',
   'ws',
   'ye',
   'yt',
   'yu',
   'za',
   'zm',
   'zr',
   'zw']
    import random
    desktop_agents = ['Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36',
             'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36',
             'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36',
             'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14',
             'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36',
             'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36',
             'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36',
             'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36',
             'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36',
             'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0']
try:
for i in range(len(paises)):
    ## json url ##
    url="https://www.espanol.skyscanner.com/g/browseservice/dataservices/browse/v3/bvweb/AR/ARS/es-MX/destinations/"+paises[i]+"/anywhere/anytime/?profile=minimalcityrollupwithnamesv2&include=image;holiday&apikey=8aa374f4e28e4664bf268f850f767535" 
    print(url)

    headers = {
        "referer": "https://www.espanol.skyscanner.com/transporte/vuelos-desde/buea/?adults=1&children=0&adultsv2=1&childrenv2=&infants=0&cabinclass=economy&rtn=0&preferdirects=false&outboundaltsenabled=false&inboundaltsenabled=false&ref=home",
        "user-agent" :random.choice(desktop_agents),
        "x-requested-with": "XMLHttpRequest"}
    response = requests.get(url,headers=headers)
    if response.status_code == 200:
        print ("ok")
    else:
        print ("error 400")
    data = response.json()
    try:
        for pais in data["PlacePrices"]:
            if pais.get('Name')== lugarDeseado:
                dprice = pais.get("DirectPrice")
                iprice = pais.get('IndirectPrice')

                if dprice != None and iprice != None:
                    print(f"{pais['Id']}  {dprice} {iprice}  {pais['Name']} ")
                # solo existe DirectPrice
                elif dprice == None:
                    print(f"{pais['Id']}  {iprice:<10}  {pais['Name']} ")
                # solo existe IndirectPrice
                else:
                    print(f"{pais['Id']}  {dprice:<10}  {pais['Name']} ")
            else:
                pass
    except:
        print("no")


else:
    i=i+1
except:
    pass

也许有一个可以更改代理的库,vpn或我不知道的任何东西,谢谢 谢谢大家

0 个答案:

没有答案