下面的代码按预期工作,但是我不得不使用xpath来查找自己,我想了解如何获取这些代码以备将来参考,以及为什么我的代码不起作用。我遵循与帮助我找到这些步骤的人完全相同的步骤,但是当我自己执行此操作时,xpath是不同的并且不起作用。我转到产品,右键单击,检查,然后右键单击html信息,然后复制xpath。我在做什么错了?
import scrapy
from scrapy import Spider
product = input("What product are you looking for? Keywords help for specific products: ")
class ProductItems(scrapy.Item):
product_title = scrapy.Field()
product_price = scrapy.Field()
product_url = scrapy.Field()
class ProductSpider(scrapy.Spider):
name = "Product_spider"
allowed_domains=['www.amazon.ca','www.newegg.ca']
start_urls = ['https://www.amazon.ca/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords='+product]
DOWNLOAD_DELAY = 10
def parse(self, response):
product_title =response.xpath('//*/div/div/div/div[2]/div[1]/div[1]/a/@title').extract()
product_price = response.xpath('//span[contains(@class,"s-price")]/text()').extract()
product_url = response.xpath('//*/div/div/div/div[2]/div[1]/div[1]/a/@href').extract()
yield{'product_title': product_title, 'product_price':product_price, 'url':product_url,}
我为product_title获得的xpath是:// * / div / div [4] / div [1] / a / @ title