Question

=“”

我有一个在本地工作的脚本，但是当我将其部署到Scrapinghub时，会出现所有错误。调试后，错误来自产生该项目。

这是我得到的错误。

ERROR   [scrapy.utils.signal] Error caught on signal handler: <bound method ?.item_scraped of <sh_scrapy.extension.HubstorageExtension object at 0x7fd39e6141d0>> Less
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/local/lib/python2.7/site-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
  File "/usr/local/lib/python2.7/site-packages/sh_scrapy/extension.py", line 45, in item_scraped
    item = self.exporter.export_item(item)
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 304, in export_item
    result = dict(self._get_serialized_fields(item))
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 75, in _get_serialized_fields
    value = self.serialize_field(field, field_name, item[field_name])
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 284, in serialize_field
    return serializer(value)
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 290, in _serialize_value
    return dict(self._serialize_dict(value))
  File "/usr/local/lib/python2.7/site-packages/scrapy/exporters.py", line 300, in _serialize_dict
    key = to_bytes(key) if self.binary else key
  File "/usr/local/lib/python2.7/site-packages/scrapy/utils/python.py", line 117, in to_bytes
    'object, got %s' % type(text).__name__)
TypeError: to_bytes must receive a unicode, str or bytes object, got int

它没有指定有问题的领域，但是通过消除过程，我意识到这是代码的这一部分：

  try:
            item["media"] = {}
            media_index = 0

            media_content = response.xpath("//audio/source/@src").extract_first()
            if media_content is not None:
                item["media"][media_index] = {}
                preview = item["media"][media_index]
                preview["Media URL"] = media_content
                preview["Media Type"] = "Audio"
                media_index += 1
        except IndexError:
            print "Index error for media " + item["asset_url"]

我整理了一些部分以使其更易于解决，但基本上这是问题所在。它对项目媒体不满意。

我是Python和Scrapy的初学者。非常抱歉，如果这是愚蠢的基本Python错误。有想法吗？

编辑：因此，从ThunderMind得到答案后，解决方案是简单地对密钥进行str（media_index）

Answer 1

是的，就在这里：

item["media"][media_index] = {}

media_index是可变的。并且键不能是可变的。阅读Python dict，以了解应将哪些内容用作键。

Scrapinghub在信号处理程序上发现错误：<bound method =“”？=“” on =“” yield =“”

1 个答案: