我无法在Pipeline scrapy中插入mysql数据库

时间:2017-08-25 03:05:02

标签: python scrapy twisted mysql-python

我在scrapy中使用Pipeline将项目插入db,但它不起作用。 我使用twisted.adbapi设置:

DB_SERVER ='MySQLdb' DB_CONNECT = {     'db':'vcinema_v4',     'user':'root',     'passwd':'my-secret-pw',     'host':'127.0.0.1',     'charset':'utf8',     'use_unicode':是的 }

类MySQLPipeline(对象):

def __init__(self):
    dbargs = settings.get('DB_CONNECT')
    db_server = settings.get('DB_SERVER')
    dbpool = adbapi.ConnectionPool(db_server, **dbargs)

    self.dbpool = dbpool

def process_item(self, item, spider):
    assert isinstance(item, CityItem)
    query = self.dbpool.runInteraction(self._conditional_insert, item)
    query.addErrback(self.handle_error)
    return item

def _conditional_insert(self, tx, item):
    print "db connected-==============>"
    tx.execute("select * from city where name=%s" % (item['name']))
    result = tx.fetchOne()
    if result:
        log.msg("Item already stored in db: %s" % item, level=log.DEBUG)
    else:
        tx.execute("""INSERT INTO city(name) VALUES(%s)""", (item['name']))
        log.msg("Item stored in db: %s" % item, level=log.DEBUG)
    print "db end"

def handle_error(self, e):
    log.err(e)

请帮帮我。提前致谢。

0 个答案:

没有答案