Question

我试图用硒python刮分页网站。我编写的代码能够从第一页提取数据，然后继续进行到第二页，但无法提取第二页和其余页的内容。

我只有第1页的结果

from selenium import webdriver
import time
browser = webdriver.Chrome(executable_path='C:\Python27\Scripts\chromedriver.exe')



browser.get("https://www.etsy.com/ca/c/jewelry/necklaces" )


posts= browser.find_elements_by_class_name("text-gray")

for post in posts:

  print post.text

for i in range(1,3):
   u=browser.get('https://www.etsy.com/ca/c/jewelry/necklaces?ref=pagination&page=%s' % str(i))

   print".................................."+ str(i)+"......................................."
time.sleep(10)   
new= u.find_element_by_class_name("text-gray")
for we in new:
   print we.text

这是我收到的错误消息：AttributeError: 'NoneType' object has no attribute 'find_elements_by_class_name

Answer 1

尝试一下：

from selenium import webdriver 

import time 

browser = webdriver.Chrome(executable_path='C:\Python27\Scripts\chromedriver.exe')
browser.get("https://www.etsy.com/ca/c/jewelry/necklaces" )
posts= browser.find_elements_by_class_name("text-gray")

for post in posts:
    print post.text

for i in range(1,3):
    gets = 'https://www.etsy.com/ca/c/jewelry/necklaces?ref=pagination&page='+str(i)
    u = browser.get(gets)
    time.sleep(10)
    new = u.find_element_by_class_name("text-gray") 
    for we in new: 
        print we.text

无法抓取循环页面（下一页）的内容

1 个答案: