Question

我已将50K对象发布到特定队列。我有一个侦听器，它选择每个对象并对其进行处理。但是很显然，处理所有5万个对象将花费更多时间。所以我想再放置3个可以并行处理这些对象的侦听器。为此，我需要再编写两个侦听器类吗？用相同的代码？那将是重复的代码。有没有一种方法可以配置所需的侦听器数量，以便在内部为同一个侦听器创建实例来处理负载？有谁能帮助我更好的方法来让3个更多的侦听器处理负载以增加处理量。 / p>

====兔子mq配置文件代码=============

@Bean
    public SubscriberGeneralQueue1 SubscriberGeneralQueue1(){
        return new SubscriberGeneralQueue1();
    }

@Bean
        public SimpleMessageListenerContainer rpcGeneralReplyMessageListenerContainer(ConnectionFactory connectionFactory,MessageListenerAdapter listenerAdapter1 ) {
            SimpleMessageListenerContainer simpleMessageListenerContainer = new SimpleMessageListenerContainer(connectionFactory);
            simpleMessageListenerContainer.setQueues(replyQueueRPC());
            simpleMessageListenerContainer.setTaskExecutor(taskExecutor());
            simpleMessageListenerContainer.setMessageListener(listenerAdapter1);
            simpleMessageListenerContainer.setMaxConcurrentConsumers(60);
            return simpleMessageListenerContainer;
        }
       @Bean
        @Qualifier("listenerAdapter1")
        MessageListenerAdapter listenerAdapter1(SubscriberGeneralQueue1 generalReceiver) {
            return new MessageListenerAdapter(generalReceiver, "receivegeneralQueueMessage");
        }

===收听者代码================

@EnableRabbit
public class SubscriberGeneralQueue1 {

     /*@Autowired
        @Qualifier("asyncGeneralRabbitTemplate")
    private AsyncRabbitTemplate asyncGeneralRabbitTemplate;*/

    @Autowired
    private ExecutorService executorService;
    @Autowired
    private GeneralProcess generalProcess;

    List <RequestPojo> requestPojoGeneral = new ArrayList<RequestPojo>();

    @RabbitHandler
    @RabbitListener(containerFactory = "simpleMessageListenerContainerFactory", queues ="BulkSolve_GeneralrequestQueue")
    public void subscribeToRequestQueue(@Payload RequestPojo sampleRequestMessage, Message message) throws InterruptedException {

        long startTime=System.currentTimeMillis();

        //requestPojoGeneral.add(sampleRequestMessage);
        //System.out.println("List size issssss:" +requestPojoGeneral.size() );
        //generalProcess.processRequestObjectslist(requestPojoGeneral);
        generalProcess.processRequestObjects(sampleRequestMessage);

        System.out.println("message in general listener is:" + sampleRequestMessage.getDistance());
        System.out.println("Message payload is:" + sampleRequestMessage);
        System.out.println("Message payload1111 is:" + message );

        //return requestPojoGeneral;

    }

}

=== simplemessagelistenercontainerFactory配置===========

 @Bean
        public SimpleRabbitListenerContainerFactory simpleMessageListenerContainerFactory(ConnectionFactory connectionFactory,
                                                                                          SimpleRabbitListenerContainerFactoryConfigurer configurer) {
            SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
            factory.setTaskExecutor(taskExecutor());
            factory.setMaxConcurrentConsumers(60);
            configurer.configure(factory, connectionFactory);
            return factory;
        }

====建议的更改=====

@RabbitHandler
    @Async
    @RabbitListener(containerFactory = "simpleMessageListenerContainerFactory", queues ="BulkSolve_GeneralrequestQueue")
    public void subscribeToRequestQueue(@Payload RequestPojo sampleRequestMessage, Message message) throws InterruptedException {

        long startTime=System.currentTimeMillis();

        //requestPojoGeneral.add(sampleRequestMessage);
        //System.out.println("List size issssss:" +requestPojoGeneral.size() );
        //generalProcess.processRequestObjectslist(requestPojoGeneral);
        generalProcess.processRequestObjects(sampleRequestMessage);

        System.out.println("message in general listener is:" + sampleRequestMessage.getDistance());
        System.out.println("Message payload is:" + sampleRequestMessage);
        System.out.println("Message payload1111 is:" + message );

        //return requestPojoGeneral;

    }


}

配置：

@Bean
        public SimpleRabbitListenerContainerFactory simpleMessageListenerContainerFactory(ConnectionFactory connectionFactory,
                                                                                          SimpleRabbitListenerContainerFactoryConfigurer configurer) {
            SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
            factory.setTaskExecutor(taskExecutor());
            factory.setMaxConcurrentConsumers(60);
            factory.setConsecutiveActiveTrigger(1);
            configurer.configure(factory, connectionFactory);
            return factory;
        }

  @Bean
        public SimpleMessageListenerContainer rpcGeneralReplyMessageListenerContainer(ConnectionFactory connectionFactory,MessageListenerAdapter listenerAdapter1 ) {
            SimpleMessageListenerContainer simpleMessageListenerContainer = new SimpleMessageListenerContainer(connectionFactory);
            simpleMessageListenerContainer.setQueues(replyQueueRPC());
            simpleMessageListenerContainer.setTaskExecutor(taskExecutor());
            simpleMessageListenerContainer.setMessageListener(listenerAdapter1);
            simpleMessageListenerContainer.setMaxConcurrentConsumers(100);
            simpleMessageListenerContainer.setConsecutiveActiveTrigger(1);
            return simpleMessageListenerContainer;
        }

Answer 1

可以通过concurrency的{{1}}选项来完成：

在RabbitMQ Client传递新消息时，在ListenerContainer中配置的TaskExecutor的线程用于调用SimpleMessageListenerContainer。如果未配置，则使用MessageListener。如果使用池执行器，请确保池大小足以处理配置的并发。使用SimpleAsyncTaskExecutor，可以直接在RabbitMQ客户端线程上调用DirectMessageListenerContainer。在这种情况下，MessageListener用于监视使用者的任务。

请从这里开始阅读：https://docs.spring.io/spring-amqp/docs/current/reference/html/_reference.html#receiving-messages

也请参见此处：https://docs.spring.io/spring-amqp/docs/current/reference/html/_reference.html#containerAttributes

concurrentConsumers（并发）-每个侦听器最初启动的并发使用者数。

更新

好的！我知道发生了什么事。

我们有这样的代码：

taskExecutor

因此，我们仅在处理第一条消息之后才检查可能的并行性。因此，您的情况将在1分钟后发生。

boolean receivedOk = receiveAndExecute(this.consumer); // At least one message received if (SimpleMessageListenerContainer.this.maxConcurrentConsumers != null) { if (receivedOk) { if (isActive(this.consumer)) { consecutiveIdles = 0; if (consecutiveMessages++ > SimpleMessageListenerContainer.this.consecutiveActiveTrigger) { considerAddingAConsumer(); consecutiveMessages = 0; } } }的另一个标志与considerAddingAConsumer()选项有关，默认情况下是这样的：

consecutiveActiveTrigger

因此，在您的情况下，允许仅精确并行显示下一条消息，您还应该配置：

private static final int DEFAULT_CONSECUTIVE_ACTIVE_TRIGGER = 10;

到/** * If {@link #maxConcurrentConsumers} is greater then {@link #concurrentConsumers}, and * {@link #maxConcurrentConsumers} has not been reached, specifies the number of * consecutive cycles when a single consumer was active, in order to consider * starting a new consumer. If the consumer goes idle for one cycle, the counter is reset. * This is impacted by the {@link #txSize}. * Default is 10 consecutive messages. * @param consecutiveActiveTrigger The number of consecutive receives to trigger a new consumer. * @see #setMaxConcurrentConsumers(int) * @see #setStartConsumerMinInterval(long) * @see #setTxSize(int) */ public final void setConsecutiveActiveTrigger(int consecutiveActiveTrigger) { Assert.isTrue(consecutiveActiveTrigger > 0, "'consecutiveActiveTrigger' must be > 0"); this.consecutiveActiveTrigger = consecutiveActiveTrigger; }。因为1仍然无法正常工作。

为了获得更好的性能，您还可以考虑将0与subscribeToRequestQueue()一起使用，以将处理过程从使用者线程真正移交给其他线程，从而避免等待一分钟来等待另一个使用者开始。

有没有办法通过使用AMQP中的配置文件为队列建立任何侦听器

1 个答案: