我正在尝试进行1000次模拟，其中包括（1）损坏道路网络，然后（2）测量由于损坏造成的交通延误。步骤（1）和（2）都涉及创建多个“地图”。在步骤（1）中，我创建了30个伤害图。在步骤（2）中，我针对这30个损坏图分别测量了交通延迟。然后，该函数返回30个损坏图上的平均流量延迟，并继续运行下一个仿真。设置的伪代码如下：
for i in range(0,1000): # for each simulation
create 30 damage maps using parallel python
measure the traffic delay of each damage map using parallel
python
compute the average traffic delay for simulation i
由于地图彼此独立，因此我在每一步都使用了并行python软件包。
该代码在第72个模拟（共1000个）中两次抛出以下错误，并在步骤（1）中停止运行，这涉及到损坏桥。
An error has occurred during the function execution
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/ppworker.py", line 90, in run
__result = __f(*__args)
File "<string>", line 4, in compute_damage
File "<string>", line 3, in damage_bridges
File "/Library/Python/2.7/site-packages/scipy/stats/__init__.py", line 345, in <module>
from .stats import *
File "/Library/Python/2.7/site-packages/scipy/stats/stats.py", line 171, in <module>
from . import distributions
File "/Library/Python/2.7/site-packages/scipy/stats/distributions.py", line 10, in <module>
from ._distn_infrastructure import (entropy, rv_discrete, rv_continuous,
File "/Library/Python/2.7/site-packages/scipy/stats/_distn_infrastructure.py", line 16, in <module>
from scipy.misc import doccer
File "/Library/Python/2.7/site-packages/scipy/misc/__init__.py", line 68, in <module>
from scipy.interpolate._pade import pade as _pade
File "/Library/Python/2.7/site-packages/scipy/interpolate/__init__.py", line 175, in <module>
from .interpolate import *
File "/Library/Python/2.7/site-packages/scipy/interpolate/interpolate.py", line 32, in <module>
from .interpnd import _ndim_coords_from_arrays
File "interpnd.pyx", line 1, in init scipy.interpolate.interpnd
File "/Library/Python/2.7/site-packages/scipy/spatial/__init__.py", line 95, in <module>
from .ckdtree import *
File "ckdtree.pyx", line 31, in init scipy.spatial.ckdtree
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/__init__.py", line 123, in cpu_count
with os.popen(comm) as p:
OSError: [Errno 35] Resource temporarily unavailable
我在带有并行python（pp）1.6.5的PyCharm虚拟环境中运行python 2.7。我的计算机运行Mac OS High Sierra 10.13.3，内存为8 GB 1867 MHz DDR3。
我认为问题出在并行python软件包或如何使用它，但否则不知如何解决此问题。人们注意到它是parallel python page上的一个错误-wkerzendorf发布在这里：
问：使用> os.system调用的作业时，出现套接字错误/内存错误
A：我发现的修复程序是使用subprocess.Popen并将> stdout，stderr弹出到subprocess.PIPE中。这是一个例子： subprocess.Popen（['ls-> rtl']，stdout = subprocess.PIPE，stderr = subprocess.PIPE，shell = True）。那>为我修复了错误。
但是，我完全不确定在哪里进行此修改。
我还读到，根据此Ghost in the Machines blog post，问题可能出在我的系统限制上。但是，当我尝试重新配置最大文件数和最大用户进程数时，在终端中收到以下消息：
Could not set resource limits: 1: Operation not permitted
我正在使用的代码相当复杂（需要运行多个输入文件），所以我担心这里无法提供一个最小的，可复制的示例。您可以下载并运行代码at this link的版本。
下面，我包括了步骤（1）的代码，在该代码中，我使用并行python创建了30个伤害图。工人人数是4。
ppservers = () #starting a super cool parallelization
# Creates jobserver with automatically detected number of workers
job_server = pp.Server(ppservers=ppservers)
print "Starting pp with", job_server.get_ncpus(), "workers"
# set up jobs
jobs = []
for i in targets:
jobs.append(job_server.submit(compute_damage, (lnsas[i%len(lnsas)], napa_dict, targets[i], i%sets, U[i%sets][:] ), modules = ('random', 'math', ), depfuncs = (damage_bridges, )))
# get the results that have already run
bridge_array_new = []
bridge_array_internal = []
indices_array = []
bridge_array_hwy_num = []
for job in jobs:
(index, damaged_bridges_internal, damaged_bridges_new, num_damaged_bridges_road) = job()
bridge_array_internal.append(damaged_bridges_internal)
bridge_array_new.append(damaged_bridges_new)
indices_array.append(index)
bridge_array_hwy_num.append(num_damaged_bridges_road)
compute_damage函数如下所示。
def compute_damage(scenario, master_dict, index, scenario_index, U):
'''goes from ground-motion intensity map to damage map '''
#figure out component damage for each ground-motion intensity map
damaged_bridges_internal, damaged_bridges_new, num_damaged_bridges = damage_bridges(scenario, master_dict, scenario_index, U) #e.g., [1, 89, 598] #num_bridges_out is highway bridges only
return index, damaged_bridges_internal, damaged_bridges_new, num_damaged_bridges
损伤桥功能看起来像这样。
def damage_bridges(scenario, master_dict, scenario_index, u):
'''This function damages bridges based on the ground shaking values (demand) and the structural capacity (capacity). It returns two lists (could be empty) with damaged bridges (same thing, just different bridge numbering'''
from scipy.stats import norm
damaged_bridges_new = []
damaged_bridges_internal = []
#first, highway bridges and overpasses
beta = 0.6 #you may want to change this by removing this line and making it a dictionary lookup value 3 lines below
i = 0 # counter for bridge index
for site in master_dict.keys(): #1-1889 in Matlab indices (start at 1)
lnSa = scenario[master_dict[site]['new_id'] - 1]
prob_at_least_ext = norm.cdf((1/float(beta)) * (lnSa - math.log(master_dict[site]['ext_lnSa'])), 0, 1) #want to do moderate damage state instead of extensive damage state as we did here, then just change the key name here (see master_dict description)
#U = random.uniform(0, 1)
if u[i] <= prob_at_least_ext:
damaged_bridges_new.append(master_dict[site]['new_id']) #1-1743
damaged_bridges_internal.append(site) #1-1889
i += 1 # increment bridge index
# GB ADDITION -- to use with master_dict = napa_dict, since napa_dict only has 40 bridges
num_damaged_bridges = sum([1 for i in damaged_bridges_new if i <= 1743])
return damaged_bridges_internal, damaged_bridges_new, num_damaged_bridges
答案 0 :(得分：0)
问题似乎是我忽略了破坏在步骤（1）和（2）中创建的服务器-一个简单的解决方法！我只是在每个步骤的末尾添加了job_server.destroy()。我目前正在运行仿真，并且已经达到了1000个中的250个。
要完全清楚，步骤（1）的代码现在为：
ppservers = () #starting a super cool parallelization
# Creates jobserver with automatically detected number of workers
job_server = pp.Server(ppservers=ppservers)
# set up jobs
jobs = []
for i in targets:
jobs.append(job_server.submit(compute_damage, (lnsas[i%len(lnsas)], napa_dict, targets[i], i%sets, U[i%sets][:] ), modules = ('random', 'math', ), depfuncs = (damage_bridges, )))
# get the results that have already run
bridge_array_new = []
bridge_array_internal = []
indices_array = []
bridge_array_hwy_num = []
for job in jobs:
(index, damaged_bridges_internal, damaged_bridges_new, num_damaged_bridges_road) = job()
bridge_array_internal.append(damaged_bridges_internal)
bridge_array_new.append(damaged_bridges_new)
indices_array.append(index)
bridge_array_hwy_num.append(num_damaged_bridges_road)
job_server.destroy()