我正在学习硒,我想获取示例网站的所有图像,该图像使用了lazyload,并且只有在图像的父元素出现在屏幕的可见范围内时,图像才会显示。
>如果图像的父元素未出现在屏幕的可见范围内,则会显示以下代码:
<a class="picture" href="http://new.qq.com/omn/20190405/20190405A0CB58.html" target="_blank"><div class="lazyload-placeholder">终于出手规范融资业务!港证监会规定最高不得超过5倍融资</div></a>
如果图像的父元素出现在屏幕的可见范围内,则会显示以下代码:
<a class="picture" href="http://new.qq.com/omn/20190405/20190405A0CB58.html" target="_blank"><img alt="终于出手规范融资业务!港证监会规定最高不得超过5倍融资" src="//inews.gtimg.com/newsapp_ls/0/8439863897_294195/0"></a>
我想控制滚动到底部的速度,以便全部显示图像。
如何控制硒中滚动到底部的速度?
我正在尝试修改window.scrollTo(0, document.body.scrollHeight);
,
但没有成功。
#coding:utf-8
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get("https://new.qq.com/rolls/?ext=news")
i = 0
while (i < 10):
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(1)
i += 1
答案 0 :(得分:1)
已更新。添加了一些代码。谢谢@Sers。
这里是示例,您如何获取新闻详细信息,例如标题和img链接,检查代码中的注释:
#coding:utf-8
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.common.action_chains import ActionChains
driver = webdriver.Chrome()
driver.get("https://new.qq.com/rolls/?ext=news")
wait = WebDriverWait(driver, 10)
# Scroll until load more button will have "没有更多了" text
while True:
driver.execute_script("arguments[0].scrollIntoView();", driver.find_element_by_id("load-more"))
if driver.find_element_by_id("load-more").text == "没有更多了":
break
# list of maps
results = []
# Gel all news and iterate
news = wait.until(ec.presence_of_all_elements_located((By.CSS_SELECTOR, "ul.list li")))
for item in news:
# scroll to each news
driver.execute_script("arguments[0].scrollIntoView();", item)
# get title
title = item.find_element_by_css_selector("h3 a").text.strip()
# wait until a.picture element will have visible img
img = wait.until(ec.visibility_of(item.find_element_by_css_selector("a.picture img")))
# add news details to the result
results.append({"title": title, "href": item.get_attribute("href"), "img": img.get_attribute("src")})
for result in results:
print(f"title: {result['title']}, img: {result['img']}")
driver.quit()