在Python中使用lambda参数对函数进行多处理

时间:2018-07-04 16:44:56

标签: python lambda python-multiprocessing

我正在尝试像这样并行化const routes: Routes = [ { path: "login", component: LoginComponent }, { path: "test", component: TestComponent }, { path: "protected", canActivate: [AuthGuardService], component: ProtectedComponent }, { path: "alsoprotected/:id", component: AlsoProtectedComponent, canActivate: [AuthGuardService], children: [ { path: "child1", component: ChildOneComponent}, { path: "child2", component: ChildTwoComponent}, { path: "child3", component: ChildThreeComponent }, { path: "child4", component: ChildFourComponent }, { path: "child5", component: ChildFiveComponent }, { path: "child6", component: ChildSixComponent }, { path: "child7", component: ChildSevenComponent } ] }, { path: "protectedsettings", canActivate: [AuthGuardService], component: SettingsComponent } ]; 中的for循环:

functionB

其中def functionA(N): M = numpy.random.rand(N,N) def xi(x): return M*x return xi def functionB(): ... xi = functionA(N) for j in range(S): recs[j], err[j] = function_to_parallelize(P, w[j], xi, L) Pwrecs是numpy数组,而err是整数。

我尝试这样做:

L

但是我收到一条错误消息,说def functionB(): ... xi = functionA(N) import multiprocessing p = multiprocessing.Pool() for j in range(S): r = p.apply_async(function_to_parallelize, (P, w[j], xi, L)) recs[j], err[j] = r.get() p.close() p.join() 不可腌制。 我认为这是因为它正在执行多处理,因此Pool使用酸洗将潜在内存分配给新进程。

然后我尝试了多线程,因为我已经知道它具有共享内存,因此通过将xi替换为p = multiprocessing.Pool(),代码可以正常工作!

除了运行不快...实际上运行得慢。从我读过的here来看,“ GIL”(全局解释器锁)仅执行单线程,所以我想可以解释这一点。

我的问题是,我如何并行化该函数?

我不想将矩阵p = multiprocessing.pool.ThreadPool()传递给M,因为它会很杂乱,并且不适用于不同的functionA。

0 个答案:

没有答案