Question

我正在尝试运行scrapy spyder，但我无法导入我的管道项

使用的图书馆

import scrapy
import json
from functools import reduce
from scrapy.selector import Selector
from anime_reviews.items import AnimereviewsItem
last_page = 1789

Spyder类是（我没有把完整的类代码）：

class AnimeReviewsSpider(scrapy.Spider):
name = 'AnimeReviews_spider'
allowed_urls =['myanimelist.net']
start_urls = ['https://myanimelist.net/reviews.php?t=anime']

def parse(self, response):
    page_urls = [response.url + "&p=" + str(pageNumber) for pageNumber in range(1, last_page+1)]
    #page_urls = ['https://myanimelist.net/reviews.php?t=anime']
    for page_url in page_urls:
        yield scrapy.Request(page_url,
            callback = self.parse_reviews_page)

def parse_reviews_page(self, response):
    item = AnimereviewsItem()
    reviews = response.xpath('//*[@class="borderDark pt4 pb8 pl4 pr4 mb8"]').extract()       #each page displays 50 reviews

    for review in reviews:
        anime_title = Selector(text = review).xpath('//div[1]/a[1]/strong/text()').extract()
        anime_url = Selector(text = 

.....
    yield item

而Items类是：

class AnimereviewsItem(scrapy.Item):
    anime_title = scrapy.Field()
    review_time = scrapy.Field()
    reviewer = scrapy.Field()
    rating = scrapy.Field()
    review_text = scrapy.Field()
    pic_url = scrapy.Field()
    anime_url = scrapy.Field()

然后，当我正在尝试AnimeReviewsItem我有“ ModuleNotFoundError：没有名为'anime_reviews'的模块

我的项目路径是：

anime_reviews / scrapy.cfg

anime_reviews / scrapy.cfg / anime_reviews /...
... / init .py，items.py，middlewares.py，pipelines.py，settings.py， pycache ，spyders / ...
... / pycache ， init .py，anime_reviews_spider.py

错误：

   runfile('C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders/anime_reviews_spider.py', wdir='C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders')
Traceback (most recent call last):

  File "<ipython-input-13-53f793bce8a2>", line 1, in <module>
    runfile('C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders/anime_reviews_spider.py', wdir='C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders')

  File "C:\Users\alexa\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "C:\Users\alexa\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders/anime_reviews_spider.py", line 13, in <module>
    from anime_reviews.items import AnimereviewsItem

ModuleNotFoundError: No module named 'anime_reviews'

没有模块命名...用于Scrapy项目管道

0 个答案: