无法从trivago.com上获取酒店的价格详细信息

时间:2019-04-09 10:35:49

标签: python web-scraping beautifulsoup

我刚开始学习bs4。我卡在下面的简单代码中,但没有得到任何输出。

我尝试使用没有运气的类来定位元素

from bs4 import BeautifulSoup
import requests

# url = "https://www.tutorialspoint.com/python3/index.htm"
url = "https://www.trivago.in/?aDateRange%5Barr%5D=2019-04-17&aDateRange%5Bdep%5D=2019-04-18&aPriceRange%5Bfrom%5D=0&aPriceRange%5Bto%5D=0&iRoomType=7&aRooms%5B0%5D%5Badults%5D=2&cpt2=64991%2F200&iViewType=0&bIsSeoPage=0&sortingId=1&slideoutsPageItemId=&iGeoDistanceLimit=20000&address=&addressGeoCode=&offset=0"
page = requests.get(url=url)

soup = BeautifulSoup(page.text,'lxml')
for s in soup.find_all("div"):
    print(s.get_text)

1 个答案:

答案 0 :(得分:2)

您可以访问json格式的响应:

import requests

url = 'https://hsg-prod-eu.nsp.trv.cloud/api/mvp/regionsearch/cep-p.json'

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36',
           'x-trv-cst': '1443703219,1459869632,1472826866,25798,27291,32046,35446,37310,39578,39329,40428,42320,42304,43107,43759,42673,44629,44982,39959,44964,43412,45265,44843,44767,45465,45433,39875,45360,44394,45698,45383,45749,45839,45429,45888,45915,45398,44278,44490,45038,46136,45800,44378,46115,44768,46187,46214,46164,46218,46090,45295,45954,46395,45695,45318'}

params = {
'csid': '8',
'ccid': 'XKx@P7@iuZXuTOXbc3fgeAAAAAY',
'adl': '3',
'crcl': '80.250824/13.052414,20000',
's': '0',
'uiv': '64991/200:1',
'tid': 'yV842GiLh1BOPXv8uYF2AHOwDO',
'sp': '20190417/20190418',
'rms': '2',
'p': 'in',
'l': 'en-IN',
'ccy': 'INR',
'accoff': '0',
'acclim': '25'}


jsonData = requests.get(url, headers=headers, params=params).json()

accommodations = jsonData['accommodations']

for each in accommodations:

    name = each['name']['value']
    site = each['deals']['bestPrice']['name']['value']
    best_price = each['deals']['bestPrice']['displayPrice']

    print ('Hotel: %s\tSite: %s  Price: %s' %(name, site, best_price))

输出:

Hotel: OYO 4355 Goodland Inn    Site: makemytrip  Price: ₹786
Hotel: Trident Chennai  Site: as_DerbyIndividual  Price: ₹10,240
Hotel: FabExpress Panasia Residency Vadapalani  Site: OYO  Price: ₹1,121
Hotel: The Belstead     Site: makemytrip  Price: ₹2,425
Hotel: MGM Beach Resorts        Site: Goibibo.com  Price: ₹5,366
Hotel: FabHotel T Nagar Chennai Stay    Site: FabHotels  Price: ₹1,987
Hotel: FabHotel Blossoms T Nagar        Site: FabHotels  Price: ₹1,780
Hotel: Treebo Trend Cine City   Site: Treebo Hotels  Price: ₹3,140
Hotel: Radisson Blu GRT Chennai Site: makemytrip  Price: ₹5,974
Hotel: Savera   Site: makemytrip  Price: ₹5,981
Hotel: The Park Chennai Site: Expedia  Price: ₹5,900
Hotel: Treebo Trend Guindy      Site: Treebo Hotels  Price: ₹2,726
Hotel: FabHotel Prime Santhome Inn Mylapore     Site: FabHotels  Price: ₹2,324
Hotel: FabHotel Emerald Manor Koyambedu Site: FabHotels  Price: ₹1,717
Hotel: The Raintree, St. Mary's Road    Site: Goibibo.com  Price: ₹5,683
Hotel: OYO 13230 ARK Residency  Site: makemytrip  Price: ₹786
Hotel: Green Meadows    Site: Goibibo.com  Price: ₹4,289
Hotel: JP Hotel Site: Goibibo.com  Price: ₹4,264
Hotel: Lemon Tree Chennai       Site: Hotels.com  Price: ₹5,353
Hotel: OYO 16596 Akash Inn Elite        Site: OYO  Price: ₹777
Hotel: Hotel Radisson Chennai City Centre, India        Site: Hotels.com  Price: ₹6,787
Hotel: Treebo Trend Sekaran     Site: Treebo Hotels  Price: ₹2,477
Hotel: Hotel Chennai Le Palace  Site: makemytrip  Price: ₹3,534
Hotel: The Lotus  Sameera       Site: makemytrip  Price: ₹2,680
Hotel: Taj Club House   Site: Booking.com  Price: ₹6,608