我正在使用scrapy并且有一个正常运行的管道,它使用sql-alchemy将记录插入到sqllite数据库中。这是有效的,但后来我添加了两个新字段:名称和地址。现在我得到了:
2016-06-14 10:12:23 [scrapy] ERROR: Error processing {'account': u'X6',
'address': u' the address',
'name': u'my name'}
Traceback (most recent call last):
File "C:\envs\virtalenvs\teat\lib\site-packages\twisted\internet\defer.py", line 588, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "C:\envs\r2\tutorial\tutorial\pipelines.py", line 54, in process_item
self.connection.execute(ins_query)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 914, in execute
return meth(self, multiparams, params)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\sql\elements.py", line 323, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1146, in _execute_context
context)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1341, in _handle_dbapi_exception
exc_info
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\util\compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\base.py", line 1139, in _execute_context
context)
File "C:\envs\virtalenvs\teat\lib\site-packages\sqlalchemy\engine\default.py", line 450, in do_execute
cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) table accounts has no column named name [SQL: u'INSERT INTO accounts (account, name, address) VALUES (?, ?, ?)'] [parameters: (u'X6', u' my name', u' the address')]
这是管道代码:
class MYPipeline(object):
def __init__(self):
_engine = create_engine("sqlite:///data.db")
_connection = _engine.connect()
_metadata = MetaData()
_stack_items = Table("accounts", _metadata,
Column("id", Integer, primary_key=True),
Column("account", Text),
Column("name", Text),
Column("address", Text))
_metadata.create_all(_engine)
self.connection = _connection
self.stack_items = _stack_items
def process_item(self, item, spider):
is_valid = True
for data in item:
if not data:
is_valid = False
raise DropItem("Missing %s!" % data)
if is_valid:
ins_query = self.stack_items.insert().values(
account=item["account"],
name=item["name"],
address=item["address"])
self.connection.execute(ins_query)
return item
奇怪的是,如果我使用以下方式运行:
scrapy crawl myspider
使用GIT-Bash(我使用win7)这可以正常工作。但是对于pycharm,我得到了上面的错误。
对于pycharm,运行配置在屏幕截图中,man.py:
from scrapy import cmdline
cmdline.execute("scrapy crawl myspider".split())
我做错了什么?
答案 0 :(得分:1)
我怀疑你正在运行两个不同的工作目录。尝试使用您在命令行上使用的工作目录配置PyCharm。或者,指定SQLAlchemy URL的绝对路径。