Question

我需要在if语句中使用if语句，我已经必须确定何时我的抓取程序单击下一步按钮，以便一旦发生这种情况我就可以做一些事情。当前的if语句仅确定页面上是否存在下一个按钮。但是我不知道如何确定何时真正单击了下一个按钮。

            # Finds next page button
            priority = response.meta['priority']
            next_page = response.xpath('//a[contains(., "- Next>>")]/@href').get()
            # If it exists and there is a next page enter if statement
            if next_page is not None:
                # Go to next page
                yield response.follow(next_page, self.parse, priority=priority, meta={'priority': priority})

Answer 1

在meta键中有一个标志，以确定该链接是否来自单击“下一步”按钮

def parse(self, response):

    if response.meta.get('isNextClicked', False):
        #Next was clicked

    # Finds next page button
    priority = response.meta['priority']    

    next_page = response.xpath('//a[contains(., "- Next>>")]/@href').get()
    # If it exists and there is a next page enter if statement
    if next_page is not None:
        # Go to next page
        yield response.follow(next_page, self.parse, priority=priority, meta={'priority': priority, 'isNextClicked': True})

下一个按钮被抓

1 个答案: