Question

意图/预期行为

从页面https://www.bezrealitky.cz/vypis/nabidka-prodej/byt/praha

返回链接的文本

以CSV格式和在shell中。

错误

我得到了一个KeyError：'title'，即使我已经在item.py项目加载器中定义了键。

完整追溯

Traceback (most recent call last):
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback
    yield next(it)
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 22, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "C:\Users\phili\Documents\Python Scripts\Scrapy Spiders\bezrealitky\bezrealitky\spiders\bezrealitky_spider.py", line 33, in parse
    yield loader.load_item()
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\loader\__init__.py", line 115, in load_item
    value = self.get_output_value(field_name)
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\loader\__init__.py", line 122, in get_output_value
    proc = self.get_output_processor(field_name)
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\loader\__init__.py", line 144, in get_output_processor
    self.default_output_processor)
  File "C:\Users\phili\Anaconda3\envs\py35\lib\site-packages\scrapy\loader\__init__.py", line 154, in _get_item_field_attr
    value = self.item.fields[field_name].get(key, default)
KeyError: 'title'

Spider.py

def解析（自我，回应）：

for records in response.xpath('//*[starts-with(@class,"record")]'):
    loader = BaseItemLoader(selector=records)
    loader.add_xpath('title', './/div[@class="details"]/h2/a[@href]/text()')
    yield loader.load_item()

Item.py - Itemloader

class BaseItemLoader(ItemLoader):
    title_in = MapCompose(unidecode)

结论

我有点不知所措，因为我认为我遵循了Scrapy手册并通过“title_in”定义了项目加载器和键，但是当我得到它的值时，我得到了KeyError。我在shell中检查Xpath提供了我想要的文本，所以至少这是有用的。希望得到一些帮助！

Answer 1

即使你使用ItemLoader，你也应该首先定义Item类，然后将它传递给项目加载器，或者将其定义为loader的属性：

class CustomItemLoader(ItemLoader):
    default_item_class = MyItem

或将其实例传递给loader的构造函数：

l = CustomItemLoader(item=Item())

否则项目加载器对项目及其字段一无所知。

Scrapy：即使定义了Key，也会出现Item Loader和KeyError

1 个答案: