在 Hotel.com 上抓取子页面时出现问题

时间:2021-06-29 14:06:52

标签: python python-3.x selenium web-scraping

这是我的脚本所在的网址:website

这是我的脚本:

import requests
from requests import get
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
import time
from selenium import webdriver
from selenium.webdriver.support.select import Select

from selenium.webdriver.support.ui import WebDriverWait     
from selenium.webdriver.common.by import By     
from selenium.webdriver.support import expected_conditions as EC

from selenium.webdriver.common.keys import Keys

PATH = "driver\chromedriver.exe"

options = webdriver.ChromeOptions() 
options.add_argument("--disable-gpu")
options.add_argument("--window-size=1200,900")
options.add_argument('enable-logging')


driver = webdriver.Chrome(options=options, executable_path=PATH)
 
wait = WebDriverWait(driver, 20)

url = 'https://fr.hotels.com/ho344560/?q-check-in=2021-06-26&q-check-out=2021-06-27&q-rooms=1&q-room-0-adults=2&q-room-0-children=0&sort-order=BEST_SELLER&WOD=6&WOE=7&JHR=1&FPQ=2&MGT=1&ZSX=0&SYE=3&YGF=1'



driver.get(url)
driver.maximize_window()
time.sleep(5)

webdriver.ActionChains(driver).send_keys(Keys.ESCAPE).perform()
#https://stackoverflow.com/questions/68178259/how-to-click-anywhere-on-a-website-to-close-an-option-selenium
time.sleep(2)

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'button[class="uolsaJ"]'))).click()

reviews = driver.find_element_by_xpath('//a[@class="WxdyDo _3ZW5IO"]')
reviews.click()

get_url = driver.current_url

#print(get_url)



ratings = []
comments = []
dates = []


#&modal=property-reviews
results = requests.get(get_url)
soup = BeautifulSoup(results.text, "html.parser")

reviews = soup.find_all('li', class_ = "_1BIjNY")

for review in reviews:
    rating = soup.find('span', class_ = '_1biq31 _11XjrQ _3yXMS-').text
    ratings.append(rating)

print(ratings)

这是我的输出:

['8,0.', '8,0.', '8,0.', '8,0.', '8,0.', '8,0.', '8,0.', '8,0.', '8,0.', '8,0.', '8,0.', '8,0.']

显然,有一些错误。你有什么想法吗?

编辑:我不明白,链接应该在这里:true website (只需点击“afficher les 21 avis”即可前往)

但它转到了之前的页面。

也许这就是我的脚本不起作用的原因?它不能转到评论并假设他在主页上?

0 个答案:

没有答案