我的蜘蛛不起作用。我用CrawlSpider类做到了

时间:2019-06-23 16:53:25

标签: python scrapy

我的蜘蛛不起作用。我希望将数据提取到一个csv文件中,但是当我打开csv文件时,它是空的。我该怎么办?

from scrapy.item import Field, Item
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
from scrapy.loader import ItemLoader
import scrapy

class misitems(Item): 
    tipo = Field()
    capacidad = Field()

class MySpider(CrawlSpider):
    name = "blog"
    allowed_domains = ["elblogdelnarco.com"]
    start_urls = ["https://elblogdelnarco.com"]

    rules = (
        Rule(LinkExtractor(restrict_xpaths=("//a[@class='next page-numbers']/@href"))),
        Rule(LinkExtractor(restrict_xpaths=("//h2[@class='title front-view-title']/a/@href")), callback='parse_item'),
    )

    def parse_item(self, response):
        item = ItemLoader(misitems(), response)
        item.add_xpath("titulo", "//h1[@class='title single-title entry-title']/text()")
        item.add_xpath("contenido", "(//div[@class='thecontent']/p/b)[1]/text()")
        yield item.load_item() 

0 个答案:

没有答案
相关问题