用硒捕获隐藏在“更多”按钮后面的文本

Question

使用Selenium for Python，我试图从此页面捕获整个描述文本：realestate.com.kh: Agile Sky Residence

在点击“阅读更多”之前：

点击“阅读更多”后：

options = webdriver.ChromeOptions()
options.add_argument('headless')
browser = webdriver.Chrome('chromedriver',chrome_options=options)
browser.maximize_window()

browser.implicitly_wait(1)
# click on "Read more", which path is //*[@id="listing-desctiption"]/div[3]/a
browser.find_elements_by_xpath('//*[@id="listing-desctiption"]/div[3]/a')[0].click()
browser.implicitly_wait(1)
# catch the full content of the unwrapped text which path is //*[@id="undefined1"]
descr = browser.find_element_by_xpath('//*[@id="undefined1"]').text
print(descr)

不幸的是，我收到以下错误：

  

selenium.common.exceptions.ElementClickInterceptedException：消息：拦截了元素单击：元素...在点（399，591）不可单击。其他元素将获得点击：...         （会话信息：headless chrome = 76.0.3809.100）

在这种情况下我无法正确进行操作吗？

Answer 1

无头模式存在问题。即使使用maximize_window，默认大小也为800x600。您可以设置自定义尺寸，在这种情况下1280x800就足够了：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions()
options.add_argument('--headless')
browser = webdriver.Chrome(options=options)
wait = WebDriverWait(browser, 5)
browser.set_window_size(1280, 800)

browser.get("https://www.realestate.com.kh/agile-sky-residence/110921/")
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".listing-more-wrapper a"))).click()
description = browser.find_element_by_class_name("description").text
print(description)

另一种解决方案是使用JavaScript获取描述文字

options = webdriver.ChromeOptions()
options.add_argument('--headless')
browser = webdriver.Chrome(options=options)
browser.maximize_window()

browser.get("https://www.realestate.com.kh/agile-sky-residence/110921/")
description = browser.execute_script("return arguments[0].textContent;",
                                     browser.find_element_by_class_name("description"))
print(description)

Answer 2

要提取整个说明文字，您需要：

• scrollIntoView()元素上带有阅读更多并在其上带有click()的文本。
• 然后，您可以使用text属性通过以下Locator Strategies阅读所需的文本：

• 代码块：

  from selenium import webdriver
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  
  chrome_options = webdriver.ChromeOptions()
  chrome_options.add_argument('--headless') 
  chrome_options.add_argument("start-maximized")
  driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.realestate.com.kh/agile-sky-residence/110921/")
  driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//a[contains(., 'Read More')]"))))
  WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(., 'Read More')]/span[@class='icon-down']"))).click()
  print(WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='Agile Sky Residence']//following::div[1]"))).text)
  driver.quit()
  

• 控制台输出：

  Agile is a Top 10 Developer in China. Over the last 27 years, Agile has excelled across eight major industries including real estate, life services, conservation, education, construction, home management, capital investment, and commercial property management.
  
  The business covers more than 100 cities around the world. Real estate development is the core industry. The nearly 200 projects have expanded to more than 70 cities across China and the world. Agile Sky Residence, a 44-storey high-rise apartment building, is located in the BKK3, one of the most prosperous areas of Phnom Penh. Located at No. 623 Monivong Blvd, it is surrounded by municipal administration, foreign affairs departments, embassies, international organizations, and many international residences.
  Agile Sky Residence is rooted in its prime location. It is convenient and transportation here is easy. What’s more, there are also high-quality schools and medical facilities that offer the best service to expats and Khmer residents in this neighbourhood. For example, it’s just a 10-minute drive to ISPP, CIS International School, BKK High School, Royal University of Law and Economics, Soviet Friendship Hospital, or First Center Polyclinic.
  Agile Sky Residence was designed by Patrick Leung, a famous interior designer from Hong Kong, who has won more than 200 international awards. It is tailored to be the ultimate residence for the new generation by using first-class brand materials and a full range of quality home appliances and furniture. It is a smart option for anyone. The unique plan and innovative facilities of Agile Sky Residence are the reason for its glory. With a beautiful 7-metre-high lobby, a massive parking area, and nine high-speed elevators, the residence delivers perfect hospitality. Agile Sky Residence has a master plan that lays out four-level sky gardens, which create a very functional space of leisure and recreation. A sky bar, sky club, cafe, a book lounge, and even a children’s playground entertain the residents here. Head to the infinity pool if you want a break. Or, for fitness buffs, go to the fitness centre or yoga area. Health and wellbeing are central aspects of living here. Agile Sky Residence hosts nearly a thousand apartments and more than 420 parking spaces.
  The apartments have a wide range of sizes from 39 to 111 square metres. Configurations range from studios to three-bedroom units. The most common type of unit is the 57-square metre one-bedroom unit. The floor plan of each unit has an excellent layout that offers a spacious and comfortable living room and a bedroom to fit the preferences of the people of Asia.