Question

我正在尝试输入getMonthEvents。但不知怎的，似乎回调从未被执行过。有任何想法吗？谢谢:)）

from scrapy.selector import Selector
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
from scrapy.http import Request
from scrapy.item import Item, Field

class EventItems(Item):
    Title = Field()
    Link = Field()
    Date = Field()
    Time = Field()
    Place =  Field()
    Description = Field()
    Program=Field()

class SpiderForHSMT(CrawlSpider):
    name = 'HMTM'
    start_urls = ['http://website.musikhochschule-muenchen.de/de/index.php?Itemid=602&id=565']
    rules =(Rule( LinkExtractor(restrict_xpaths=('//div[@id="VER_2013_DISPLAYSEARCHRESULTS"]/table[1]/tr[3]'), tags=('a',), attrs=('href',)), callback = 'parseMonth'), )

    def parseMonth(self, response):
        request = Request(response.url, callback = self.getMonthEvents)
        yield request

    def getMonthEvents(self, response):
        print(response.url)

Answer 1

当您在parseMonth中复制请求时，请求将作为重复项(see documentation)进行过滤。将<{1}}添加到您的请求中，以便不会对其进行过滤。

dont_filter=True

Python / Scrapy：没有回调的yield请求

1 个答案: