如何使用SQLAlchemy将行从本地SQLite DB迁移到远程MySQL数据库?

时间:2018-04-19 03:25:41

标签: python mysql sqlite sqlalchemy google-cloud-platform

我正在使用SQLAlchemy来管理2个数据库。一个是本地SQLite数据库,另一个是Google云平台上的远程MySQL数据库,它们都共享相同的模式。我正在尝试将SQLite DB中包含的行附加到MySQL DB。我目前有一个文件定义我的类,如下所示:

SensorCollection.py

class Readings(Base):
    __tablename__ = 'readings'

    time = Column(String(250), primary_key=True)
    box_name = Column(String(250))
    FS = Column(Numeric)
    IS = Column(Numeric)
    VS = Column(Numeric)
    CO = Column(Numeric)
    TVOC = Column(Numeric)
    cTemp = Column(Numeric)
    fTemp = Column(Numeric)
    humidity = Column(Numeric)
    pressure = Column(Numeric)

然后我有另一个声明我的Base,Session和引擎的文件:

base.py

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

engine = create_engine('sqlite:///%s' % 'db/db.db')
Session = sessionmaker(bind=engine)
Base = declarative_base()

最后是我的实际代码:

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.schema import Table
from SensorCollection import Readings
from sqlalchemy.sql.expression import insert

# uploads MySQL database to Google cloud server
# uses ORM to ensure the databases can communicate with each other
user = ***
password = ***
host = ***
port = ***
dbName = ***
instanceConnectionName = ***
# create the engine for the local db
engineLocal = create_engine('sqlite:///%s' % 'db/db.db')
BaseLocal = Readings(None, None, None, None, None, None, None, None, None, None, None)

# create the engine for the Google db
engineGoogle = create_engine('mysql+mysqldb://%s@%s:%s/%s?unix_socket=/cloudsql/%s' % (user, host, port, dbName, instanceConnectionName))
SessionGoogle = sessionmaker(bind=engineGoogle)
BaseGoogle = Readings(None, None, None, None, None, None, None, None, None, None, None)

# create the local db
BaseLocal.metadata.create_all(engineLocal)

# create the Google db and start the session
BaseGoogle.metadata.create_all(engineGoogle)
sessionGoogle = SessionGoogle()

# create table objects for each db
t1 = Table('readings', BaseLocal.metadata, autoload=True, autoload_with=engineLocal)
t2 = Table('readings', BaseGoogle.metadata, autoload=True, autoload_with=engineGoogle)

# the first subquery, select all ids from SOME_TABLE where some_field is not NULL
s1 = t1.select()

# the second subquery, select all ids from SOME_TABLE where some_field is NULL
s2 = t2.select()

# union s1 and s2 subqueries together and alias the result as "alias_name"
q = s1.union(s2)

insert(t2, s1)
sessionGoogle.query(q)
# sessionGoogle.add_all(s1)

# commit changes and close the session
sessionGoogle.commit()
sessionGoogle.close()

目前,SQLite数据库存在得很好并且远程服务器已经设置好,但是代码在连接到Google时甚至没有创建表。它似乎正确连接,因为如果我改变任何服务器细节,我会收到MySQL错误。它也没有给我任何错误。它编译并运行,但不附加数据甚至创建表。这是我第一次对任何类型的数据库进行破解,所以我确信它充满了错误并且根本不是优雅的。任何帮助将不胜感激。

修改 djkern帮助我创建了表格,并且我已经更新了我的代码。不过,我仍然没有得到数据。

2 个答案:

答案 0 :(得分:1)

在您的代码中,您有:

# create the Google db and start the session
BaseLocal.metadata.create_all(engineLocal)
sessionGoogle = SessionGoogle()

你的意思是这个吗?

# create the Google db and start the session
BaseGoogle.metadata.create_all(engineGoogle)
sessionGoogle = SessionGoogle()

答案 1 :(得分:1)

需要添加两个类,

class ReadingsLocal(BaseLocal):

class ReadingsGoogle(BaseGoogle):

它们应与

位于同一文件中
class Readings(Base):

还必须创建新的Base,引擎和Session变量。然后,代码应如下所示:

# create the local db
BaseLocal.metadata.create_all(engineLocal)
sessionLocal = SessionLocal()

# create the Remote db and start the session
BaseGoogle.metadata.create_all(engineGoogle)
sessionGoogle = SessionGoogle()

for reading in sessionLocal.query(ReadingsLocal):
    # reading is already attached to a session, so a tmp variable must be created to avoid conflicts
    tmp = ReadingsGoogle(reading.time, reading.box_name, reading.FS, reading.IS, reading.VS, reading.CO, reading.TVOC,
               reading.cTemp, reading.fTemp, reading.humidity, reading.pressure)
    sessionGoogle.add(tmp)

# commit changes and close the session
sessionGoogle.commit()
sessionGoogle.close()