rabbitmq过滤器工作队列

Question

我一直在阅读rabbitMQ教程，我正在寻找一些关于我应该使用的设置的帮助。

我有一个任务1-50的列表，我希望在一组4台计算机上运行一次（并且只运行一次），每台计算机都运行一个工作程序。我在https://www.rabbitmq.com/tutorials/tutorial-two-python.html

设置了类似于教程2的模板

并非所有计算机都可以运行所有任务（他们还没有安装软件）

我想要实现的是允许过滤发送给工作人员的任务的设置。

我阅读了有关如何在广播情况下使用路线实现此目的的教程，但是我并没有完全掌握我需要做什么才能将其映射回类似于教程2的简单推送模型（因为我没有＆＃ 39;我想广播这些工作。）

在某些方面，我希望能够根据负载动态扩展每个方框上的工作人员数量。

我应该使用的最佳模型是什么？是否有任何好的教程或可以推荐的内容来了解​​这种方法？

干杯， 罗布

Answer 1

RabbitMQ doesn't provide a way to selectively consume messages from a queue. A consumer on a queue will always have a chance to receive any given message in that queue. Therefore, you have to pre-filter the messages in to queues for the specific type of work to be done. Once you've done that, your message consumers only consume from the queues for the type of work they can handle.

Say you have 3 types of work to do:

• JobA
• JobB
• JobC

If you try to push messages for all three types of work in to a single queue, then your consumer has to be smart about which ones it can handle. This doesn't work. Your consumer would have to nack the message back on to the queue if it can't handle it. But there's no guarantee that your message will be picked up by a different consumer that can handle it. It may go back to the same consumer which would then nack it back on to the queue again.

This is the selective consumer anti-pattern in RabbitMQ.

Instead, you need to pre-filter your messages in to queues for specific types of work. You do this through the use of routing keys in the exchange -> queue bindings.

Using the three job types above, you could have a setup like this:

| exchange | routing key | queue |
| -------- | ----------- | ----- |
| jobs     | job.a       | job.a |
| jobs     | job.b       | job.b |
| jobs     | job.c       | job.c |

Your code that consumes these messages needs to know which type of job it can handle. Then, it only subscribes to the queue for that type of work.

Say you have 2 computers that are message consumers. Computer 1 can handle JobA and JobB. Computer 2 can handle JobB and JobC. In this scenario, you end up with 1 computer handling JobA, 2 computers handling JobB and 1 computer handling JobC. There are only 2 computers total, but each of them handles multiple jobs... only the jobs they know how to handle, though.

You guarantee Computer1 only gets JobA and JobB by only having it subscribe to the queue for job.a and job.b. The same goes for any other consumer in your system.

Having done this, scaling the number of workers is easy. You need more workers on JobA? No problem. Just add another consumer of the job.a queue.

Hope that helps!