I would like to improve the speed of this calculation by using the multiprocessing. But, I don't understand why my implementation (the second solution) doesn't work... Python "crash" (Python left in a unforeseen way - see below for a part of the report) without exception, or error
# -*- coding: utf-8 -*-
import time
import numpy as np
import multiprocessing as mp
a = 1.1
b = 0.1
d = 2
M = 1000
N = 500
P = 100
tab = np.random.random_sample([N,M])*1000
vectors = np.random.random_sample([P,M])*100
coef = np.random.random_sample([1,P])
def sol2(i):
return np.dot(coef,(a + b*np.dot(tab,vectors.T).T)**d)
if __name__ == '__main__':
#sol1
start_time = time.time()
for i in range(1000):
res = np.dot(coef,(a + b*np.dot(tab,vectors.T).T)**d)
print("--- %s seconds ---" % (time.time() - start_time))
#sol2
start_time = time.time()
pool = mp.Pool()
asyncResult = pool.map_async(sol2, xrange(1))# 1 to test
res = asyncResult.get()
print("--- %s seconds ---" % (time.time() - start_time))
Apple Report: Process: Python [96770] Path: /usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python Identifier: Python Version: 2.7.9 (2.7.9) Code Type: X86-64 (Native) Parent Process: Python [96766] Responsible: iTerm [96704] User ID: 501
Date/Time: 2015-05-24 23:00:03.942 +0200 OS Version: Mac OS X 10.10.3 (14D136) Report Version: 11 Anonymous UUID: AC6120B2-F5B6-D63F-641A-81A79258B4D8
Sleep/Wake UUID: 7AF8B0D6-5E2E-4794-84BE-0DABEE821CB9
Time Awake Since Boot: 150000 seconds Time Since Wake: 130000 seconds
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000110
VM Regions Near 0x110: --> __TEXT 0000000101cb0000-0000000101cb2000 [ 8K] r-x/rwx SM=COW /usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
Application Specific Information: crashed on child side of fork pre-exec ...