我正在使用scrapy python来抓取特定网站。该网站的格式如下:
http://www.example.com/s/ref=lp_1805560031_pg_4?rh=n%3A976419031%2Cn%3A%21976420031%2Cn%3A1389401031%2Cn%3A1389432031%2Cn%3A1805560031& 页= 4 &安培,即= UTF8&安培; QID = 1400668237
如果我想从第1页到第30页中删除,我如何在这种情况下包括处理分页;
我试过了:
class MySpider(BaseSpider):
start_urls = ['http://www.example.com/s/ref=lp_1805560031_pg_4?rh=n%3A976419031%2Cn%3A%21976420031%2Cn%3A1389401031%2Cn%3A1389432031%2Cn%3A1805560031&page=%s&ie=UTF8&qid=1400668237' % page for page in xrange(1,30)]
但它不起作用
编辑:我们仅将域名作为example.com用于问题目的
答案 0 :(得分:4)
这应该对你有用
start_urls = ['http://www.example.com/s/ref=lp_1805560031_pg_4?rh=n%3A976419031%2Cn%3A%21976420031%2Cn%3A1389401031%2Cn%3A1389432031%2Cn%3A1805560031&page={0}&ie=UTF8&qid=1400668237'.format(page) for page in xrange(1,30)]