Question

在scrapy云中运行我的代码时出现这个奇怪的错误。不知道如何调试它。蜘蛛代码中没有对行的引用。我假设它是关于保存项目和smth一般，因为没有指示url。此外，蜘蛛运行正常，并在此错误后提供结果。任何帮助表示赞赏。

    [scrapy.utils.signal] Error caught on signal handler: <bound method ?.item_scraped of <sh_scrapy.extension.HubstorageExtension object at 0x7fcdc33abf50>> Less
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/local/lib/python2.7/site-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
  File "/usr/local/lib/python2.7/site-packages/sh_scrapy/extension.py", line 45, in item_scraped
    item = self.exporter.export_item(item)
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 304, in export_item
    result = dict(self._get_serialized_fields(item))
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 75, in _get_serialized_fields
    value = self.serialize_field(field, field_name, item[field_name])
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 284, in serialize_field
    return serializer(value)
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 290, in _serialize_value
    return dict(self._serialize_dict(value))
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 300, in _serialize_dict
    key = to_bytes(key) if self.binary else key
  File "/usr/local/lib/python2.7/site-packages/scrapy/utils/python.py", line 117, in to_bytes
    'object, got %s' % type(text).__name__)
TypeError: to_bytes must receive a unicode, str or bytes object, got instance

这是我收益的项目：

  {response.meta['url'] : {"rss_categories": [], "int_links" : int_links, "ext_links" : ext_links, "all_links" : len(all_links), "email" : email, "url" : url, "social_media":[{"twitter" : twitter,"facebook" : facebook, "instagram" : instagram, "pinterest" : pinterest, "youtube" : youtube }],"rss_atom" : rss_atom, "title" : title, "MetaDescription" : MetaDescription, "descr" : descr, "keywords" :[{}]}}

信号处理程序捕获到错误：TypeError：to_bytes必须接收unicode获取的实例

0 个答案: