当单个进程未执行Python多处理会引发错误

时间:2020-03-18 05:13:47

标签: python postgresql multiprocessing

运行环境:Mac OS
数据库:Postgres
Python:3.6

我正在尝试通过多处理来加快计算速度。

让我总结一下这个问题,以免您感到困惑。

保持原状

users = [UserModel1, UserModel2..UserModel1000]
calculator_fomular = [formular1<Instance of BaseCalculator>, formular2...formular100]

目前,每个用户都从列表中弹出,并发送到for循环中,并按以下方式进行计算。

for user in users:
    for formula in calculator_formula:
        result = formula(user.score)
        session.add(result)
session.commit()

因此,这需要很长时间,并且仅使用单个线程即可计算整个公式中的所有用户。当前工作正常。

但是当我尝试通过将公式像这样分成几堆来进行多处理时,

for user in users:
    for formula in calculator_formula[(i-1)*10:i*10]: # variable 'i' is differently assigned by Parent.
        result = formula(user.score)
        session.add(result)
session.commit()

# users is fetched from database. Each process has its own db connection and fire the query to get user lists.
# formula is given from its parent.
# Both are the same at single process.

我应该做些不同的事情吗?

0 个答案:

没有答案