我已经编写了一个简单的蜘蛛程序来检索报价信息:
import scrapy
class GoodReadsSpider(scrapy.Spider):
#identity
name = 'goodreads'
#requests
def start_requests(self):
url = "https://www.goodreads.com/quotes?page=1",
yield scrapy.Request(url=url, callback= self.parse)
#response
def parse(self, response):
for quote in response.selector.xpath("//div[@class='quote']"):
yield {
'text': quote.xpath(".//blockquote[@class='quoteBody']/text()[1]").extract_first(),
'author': quote.xpath(".//span[@class='quoteAuthor']/text()").extract_first(),
'tag': quote.xpath(".//div[@class='quoteTags']/a/text()").extract(),
}
运行它时,出现以下错误:
Request url must be str or unicode, got %s:' % type(url).__name__
有人知道为什么吗?
答案 0 :(得分:1)
在start_requests
的{{1}}中,您在行尾加了逗号,因此它认为url
是一个元组。
url