我是Python多重处理的新手。我有一些班级,每个班级代表一些要完成的任务。下面是一小段类结构的片段
class TaskA():
parallel_limit = 2 # Max number of process in Pool
time_limit = 1 # Denotes time taken to perform some operation
queue_name = 'read_here' # RabbitMQ queue to provide data for consumption
# Starts a pool of processes listening on queue
def start_process_pool(self):
pool = multiprocessing.Pool(processes=self.parallel_limit)
for i in range(0, self.parallel_limit):
pool.apply_async(self.consume)
try:
while True:
continue
except KeyboardInterrupt:
print(' [*] Exiting...')
pool.terminate()
pool.join()
# Reads from queue and performs some operation
def consume(self):
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.queue_declare(queue=self.queue_name, durable=True)
channel.basic_consume(queue=self.queue_name, on_message_callback=self.perform_operation, auto_ack=True)
print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()
# Performing some operation
def perform_operation(self, ch, method, properties, body):
print(" [x] %r received %r" % (multiprocessing.current_process(), body,))
time.sleep(self.time_limit)
ch.basic_ack(delivery_tag=method.delivery_tag)
我能够在Pool中运行此任务。当我有多个任务并且需要在池中运行每个任务时,就会出现问题。我正在考虑为每个任务创建一个进程,然后从中开始一个进程池,但是无法弄清楚该如何做。任何贡献都会有很大帮助。