我的主要目标是从父进程中打开30个子进程,然后从那30个子进程中的每个进程中打开未知数量的新进程。我要给那些新的子进程中的一些位置数据打电话给redis,但我不确定要打电话多少次,这应该是100次或超过1000次。当我打电话超过1000次时,我正在跨页限制,错误是:
OSError: [Errno 24] Too many open files
我不想在生产服务器上手动增加页面限制。我想限制流程的创建,因此它决不能打开超过1000个连接。
这是我的模板代码:
import multiprocessing
import time
from multiprocessing.dummy import Pool
from random import randint
class MultiProcessing():
def second_calculation(self, index1, index2):
random = randint(1, 10)
time.sleep(random)
print("Slept for: {} seconds".format(random))
print("Call done: index: {} | index2: {}".format(index1, index2))
def calculation(self, index):
child_process = list()
random = randint(1, 5)
time.sleep(random)
print("Slept for : {} seconds".format(random))
counter = 0
for i in range(0, 1500):
counter += 1
new_child_process = multiprocessing.Process(target=self.second_calculation, args=(index, counter))
child_process.append(new_child_process)
new_child_process.start()
for process in child_process:
process.join()
print("Request done: {}".format(index))
if __name__ == '__main__':
index = 0
parent_process = list()
m = MultiProcessing()
for i in range(0, 30):
index += 1
print("Index: {}".format(index))
new_process = multiprocessing.Process(target=m.calculation, args=(index,))
parent_process.append(new_process)
new_process.start()
for process in parent_process:
process.join()
谢谢。