在Mac OS X上运行Scrapy教程时出现异常

时间:2015-12-11 18:17:49

标签: macos sqlite scrapy

我正在尝试学习Scrapy,并且我在Mac OS X 10.11.2上运行它。 我按照教程,下载教程文件并创建了一个新的Spider文件,如下所述:http://doc.scrapy.org/en/1.0/intro/tutorial.html

当我尝试运行蜘蛛时,我得到以下异常:

2015-12-11 19:04:05 [scrapy] INFO: Scrapy 1.0.3 started (bot: tutorial)
2015-12-11 19:04:05 [scrapy] INFO: Optional features available: ssl, http11
2015-12-11 19:04:05 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'BOT_NAME': 'tutorial'}
2015-12-11 19:04:06 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsole, LogStats, CoreStats, SpiderState
Unhandled error in Deferred:
2015-12-11 19:04:06 [twisted] CRITICAL: Unhandled error in Deferred:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 150, in _run_command
cmd.run(args, opts)
File "/usr/local/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 57, in run
self.crawler_process.crawl(spname, **opts.spargs)
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 153, in crawl
d = crawler.crawl(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1274, in unwindGenerator
return _inlineCallbacks(None, gen, Deferred())
--- <exception caught here> ---
File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
result = g.send(result)
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 71, in crawl
self.engine = self._create_engine()
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 83, in _create_engine
return ExecutionEngine(self, lambda _: self.stop())
File "/usr/local/lib/python2.7/site-packages/scrapy/core/engine.py", line 64, in __init__
self.scheduler_cls = load_object(self.settings['SCHEDULER'])
File "/usr/local/lib/python2.7/site-packages/scrapy/utils/misc.py", line 44, in load_object
mod = import_module(module)
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/usr/local/lib/python2.7/site-packages/scrapy/core/scheduler.py", line 6, in <module>
from queuelib import PriorityQueue    
File "/usr/local/lib/python2.7/site-packages/queuelib/__init__.py", line 1, in <module>
from queuelib.queue import FifoDiskQueue, LifoDiskQueue
File "/usr/local/lib/python2.7/site-packages/queuelib/queue.py", line 5, in <module>
import sqlite3 
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/dbapi2.py", line 28, in <module>
from _sqlite3 import *
 exceptions.ImportError: No module named _sqlite3
2015-12-11 
19:04:06 [twisted] CRITICAL: 

最初我认为我的sqlite安装有问题,但是如果我从命令行运行命令sqlite3,它似乎正在工作。有什么想法吗?

1 个答案:

答案 0 :(得分:1)

显然这是一个已知的自制问题,请参阅here

我认为你应该重新安装自制软件的sqlite和python