多处理如何关闭每个线程的数据库连接

时间:2017-04-26 14:26:35

标签: python python-multiprocessing

我正在为每个线程创建一个数据库连接,并且想知道当我完成时关闭这些连接的最佳方法是什么。

这是我的代码,我初始化了我的数据库连接和调用池

class Processor(object):
    def __init__(self, DSN):
        self.dsn0= DSN[0]
        self.dsn1= DSN[1]
        self.conn0 = None
        self.conn1 = None


    def __call__(self, data):
        if(self.conn0 is None and self.conn1 is None):
           self.conn0 = psycopg2.connect(self.dsn0)
           self.conn0Curs = self.conn0.cursor()
           self.conn1 = psycopg2.connect(self.dsn1)
           self.conn1Curs = self.conn1.cursor()
       sql = generateQuery(*data)
       print(sql)



if __name__ == '__main__':

    pool = ThreadPool()
    pool.map(Processor([DSN1, DSN2]), batches)
    pool.close()
    pool.join()

1 个答案:

答案 0 :(得分:0)

我建议您使用Context Manager而不是在线程中打开连接。它的主要优点是即使出现错误也能确保关闭连接。

class Processor(object):
    def __init__(self, DSN):     
        self.dsn0= DSN[0]
        self.dsn1= DSN[1]
        self.conn0 = None
        self.conn1 = None 
        self.curs0 = None
        self.curs1 = None

    def __enter__(self):
        self.conn0 = psycopg2.connect(self.dsn0)
        self.conn1 = psycopg2.connect(self.dsn1)
        self.curs0 = self.conn0.cursor()
        self.curs1 = self.conn1.cursor()
        return self

    def __exit__(self, *_):
        """Close the connections."""
        self.conn0.close()
        self.conn1.close()

    def process(self, data):
        sql = generateQuery(*data)
        print(sql)


if __name__ == '__main__':
    with Processor([DSN1, DSN2]) as processor:
        pool = ThreadPool()
        pool.map(processor.process, batches)
        pool.close()
        pool.join()