错误:PyMySQL无法在scrapy的蜘蛛中工作

时间:2018-06-23 06:23:33

标签: python-3.x scrapy scrapy-spider pymysql

当我在python的scrapy项目中导入pyMysql库时,出现错误,找不到模块。我想问一下如何在scrapy项目的python文件中导入pyMysql库。当我以简单的python导入pyMySQL时,它可以正常工作。

在蜘蛛中,它是由命令“ genspider spider_name(url)”生成的,我使用了此代码,该代码给出了错误。

enter code here

import scrapy
from amazon.items import AmazonItem
import pymysql


class AmazonProductSpider(scrapy.Spider):
    name = "AmazonDeals"
    allowed_domains = ["amazon.com"]

# Use working product URL below
start_urls = [
    "https://www.amazon.com/gp/product/B01IO0QWJA","https://www.amazon.in/Mi-Redmi-5-Gold-32GB/dp/B0756RF9KY"
]

def parse(self, response):
    items = AmazonItem()
    title = response.xpath('//h1[@id="title"]/span/text()').extract()
    sale_price = response.xpath('//span[contains(@id,"ourprice") or contains(@id,"saleprice")]/text()').extract()
    category = response.xpath('//a[@class="a-link-normal a-color-tertiary"]/text()').extract()
    availability = response.xpath('//div[@id="availability"]//text()').extract()
    items['product_name'] = ''.join(title).strip()
    items['product_sale_price'] = ''.join(sale_price).strip()
    items['product_category'] = ','.join(map(lambda x: x.strip(), category)).strip()
    items['product_availability'] = ''.join(availability).strip()
    yield items

1 个答案:

答案 0 :(得分:1)

检查是否安装了PyMySQL模块:

python -c "import pymysql"

如果一切正常,此命令将不返回任何内容。

如果返回 ModuleNotFoundError ,则使用pip install the module

pip install PyMySQL