我使用scrapyd来运行scrapy作业,有错误:
我不知道为什么会发生这种情况?
请帮我。
ERROR: Error caught on signal handler: <bound method ?.item_scraped of <scrapy.contrib.feedexport.FeedExporter object at 0x7fbed853fe10>>
Traceback (most recent call last):
File "/home/user1/env/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/core/scraper.py", line 215, in _itemproc_finished
item=output, response=response, spider=spider)
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/signalmanager.py", line 23, in send_catch_log_deferred
return signal.send_catch_log_deferred(*a, **kw)
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/utils/signal.py", line 53, in send_catch_log_deferred
*arguments, **named)
--- <exception caught here> ---
File "/home/user1/env/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 139, in maybeDeferred
result = f(*args, **kw)
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/xlib/pydispatch/robustapply.py", line 54, in robustApply
return receiver(*arguments, **named)
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/contrib/feedexport.py", line 190, in item_scraped
slot.exporter.export_item(item)
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/contrib/exporter/__init__.py", line 111, in export_item
self.file.write(self.encoder.encode(itemdict))
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/utils/serialize.py", line 89, in encode
return super(ScrapyJSONEncoder, self).encode(o)
File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
File "/home/user1/env/local/lib/python2.7/site-packages/scrapy/utils/serialize.py", line 109, in default
return super(ScrapyJSONEncoder, self).default(o)
File "/usr/lib/python2.7/json/encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
exceptions.TypeError: <Store: Store object> is not JSON serializable
items.py:
import scrapy
from scrapy.contrib.djangoitem import DjangoItem
from place.models import Address
class AddressItem(DjangoItem):
django_model = Address
pipeline.py
class FoodPipeline(object):
def process_item(self, item, spider):
if isinstance(item, AddressItem):
product = item.save(commit=False)
product_exist = Address.objects.filter(add=product.add)
if product_exist:
product.id = product_exist[0].id
product.save()
return item