使用SQLAlchemy

时间:2017-05-03 20:22:33

标签: python python-3.x sqlalchemy python-multiprocessing

我是使用SQLAlchemy的新手,我正在研究一个复杂的ETL过程,所以我做了以下简化的代码:

module1.py

class Foo:
    def foo_method(self):
        # doing stuff with database

module2.py

class Bar:
    def bar_method(self):
        # doing stuff with database

main_script.py

from module1 import Foo
from module2 import Bar

def run():
    with Pool(processes = num_workers) as pool:
        responses = [pool.apply_async(some_func, (param)) for param in params]
        for response in responses:
            response.get()


def some_func(param):
    engine = create_engine(connection_string, echo=True)
    Session = scoped_session(sessionmaker(bind=engine))
    session = Session()    
    # Start doing some stuff with database
    foo = Foo()
    foo.foo_method()

    bar = Bar()
    bar.bar_method()

所以我有一个带有工作进程的池。当我调用main_script.run()时,每个worker在some_func内创建一个数据库会话。我的问题是如何在module1和module2中为每个worker使用相同的会话而不通过param将会话传递给每个方法?我应该在每个模块/文件中添加以下行吗?

    engine = create_engine(connection_string, echo=True)
    Session = scoped_session(sessionmaker(bind=engine))
    session = Session()   

0 个答案:

没有答案