在Python中使用Selenium时无法抓取评论

Question

我想从以下网站抓取评论：https://www.sephora.com/product/the-porefessional-face-primer-P264900。这是检查评论时发现的语法示例：

<div class="css-7rv8g1 " data-comp="Ellipsis Box ">So good! This primer smooths my skin and blurs my pores so well! But, it is pretty mattifying so if you want a dewy look, this might not be for you.</div>

我尝试了以下代码，该代码返回一个空列表：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('/…/chromedriver')
url = 'https://www.sephora.com/product/the-porefessional-face-primer-P264900'
driver.get(url)
reviews = driver.find_elements_by_xpath("//div[@id='ratings-reviews']//div[@data-comp='Ellipsis Box']")

我尝试在find_elements上调用其他driver方法，但未成功。我还尝试了this answer概述的解决方案，但通过运行以下代码得到了TimeoutException：

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver.get(url)
driver.execute_script("arguments[0].scrollIntoView(true);", WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.XPATH, "//div[@id='tabpanel0']/div//b[contains(., 'What Else You Need to Know')]"))))
reviews = WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@data-comp='GridCell Box']//div[@data-comp='Ellipsis Box']")))

如何使用Selenium从丝芙兰网站上的此页面抓取评论？

Answer 1

您需要在xpath中添加一个空格。您应该将“省略号框”设置为“省略号框”

//div[@id='ratings-reviews']//div[@data-comp='Ellipsis Box ']

我能够使用更正后的xpath找到6个元素。