如何网页抓取下一页

Question

我正在尝试从此网站https://publishednotices.asic.gov.au/browsesearch-notices/抓取数据

问题在于，没有下一页，并且网站是动态的。有人可以帮我吗？

Answer 1

尝试一下：

from selenium import webdriver
import time


browser = webdriver.Firefox()

# open url
browser.get("https://publishednotices.asic.gov.au/browsesearch-notices/")

for page in range(3,7):
    browser.find_element_by_css_selector(f"td:nth-child({page}) a").click()

    # get data

    time.sleep(1)

tmp = True
while tmp:
    for page in range(4, 9):
        try:
            browser.find_element_by_css_selector(f"td:nth-child({page}) a").click()
            time.sleep(1)
        except:
            tmp = False 
            

Answer 2

您可以获得下一个页码元素，然后使用element.click()

单击它