并发生产者在Rabbitmq中无限期地阻止

时间:2015-05-30 09:12:18

标签: java performance rabbitmq messaging spring-rabbit

我在使用spring-rabbit-1.3.9.RELEASE库进行Rabbitmq 3.3.5的POC时观察到奇怪的行为

当我开始一个生产线程时,事情顺利进行。但是如果同时启动多个线程,只有其中一个线程完成,所有其他线程无限期地被阻塞,即使队列变空也是如此。

rabbitmqctl list_connections进行监控时,被阻止线程的连接状态仍然在运行。应该注意的是,当生产者阻止时,或者在整个运行期间的任何其他时间,都没有警报。

我还观察到,如果我在每次发送后进行1毫秒的睡眠,问题就会消失。

所以,我有这些问题

  1. rabbitmq不支持并发生产者,高价出版吗?
  2. 即使连接确实被阻止,为什么它不会显示在rabbitmqctl list_connections中?
  3. 为什么它们会无限期阻塞而不能恢复乳清队列变空?
  4. 代码

        public static void main(String[] argv) throws java.io.IOException, InterruptedException {
            init();
            PocConfig config = new PocConfig();
            int threadCount = config.getThreadCount();
            final int eventsPerThread = config.getEvents() / threadCount;
            final long sleep = config.getSleep();
    
            System.out.println("Start producer with configuration [threadCount=" + threadCount + ", events=" + eventsPerThread + ", sleep="
                + sleep + "]");
    
            ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
            for (int i = 0; i < threadCount; i++) {
                final int threadId = i;
                executorService.submit(new Runnable() {
                    public void run() {
                        produce(eventsPerThread, sleep, threadId);
                    }
                });
            }
            waitAndTearDown(executorService);
        }
    
        private static void produce(int events, long sleep, int threadId)     {
            long start = System.currentTimeMillis();
            for (int index = 1; index <= events; index++) {
                try {
                    byte[] message = messageFactory.createTestMessage(index);
                    amqpTemplate.convertAndSend(QUEUE_NAME, message);
                    if (sleep > 0) {
                        Thread.sleep(sleep);
                    }
                } catch (Exception e) {
                    LOG.error("Error", e);
                }
            }
            long time = System.currentTimeMillis() - start;
            System.out.println("Producer:" + threadId + " finished, events: " + events + ", Time(s): " + time / 1000 + ", tps: " + (events * 1000) / time);
        }
    

    Spring配置

    <bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
        <property name="addresses" value="${addresses}" />
        <property name="username" value="${user}" />
        <property name="password" value="${passwd}" />
        <property name="cacheMode" value="CONNECTION" />
        <property name="connectionCacheSize" value="${threads}" />
        <property name="channelCacheSize" value="10" />
    </bean>
    
    <rabbit:template id="template" connection-factory="connectionFactory"
        exchange="testExchange" routing-key="testQueue"/>
    

1 个答案:

答案 0 :(得分:1)

没有什么我能想到会阻挡,所以我只是跑了你的考试;没有问题:

Start producer with configuration [threadCount=5, events=10, sleep=0]
Producer:2 finished, events: 1000, Time(s): 0, tps: 4405
Producer:3 finished, events: 1000, Time(s): 0, tps: 4132
Producer:1 finished, events: 1000, Time(s): 0, tps: 4048
Producer:0 finished, events: 1000, Time(s): 0, tps: 3968
Producer:4 finished, events: 1000, Time(s): 0, tps: 3952

是什么让你觉得他们被封锁了?

进行线程转储(例如使用jstack)以查看线程正在做什么。

修改

即使有1M条消息和CacheMode CONNECTION ...

,我仍然无法重现它
Start producer with configuration [threadCount=5, events=200000, sleep=0]
Producer:0 finished, events: 200000, Time(s): 50, tps: 3959
Producer:3 finished, events: 200000, Time(s): 53, tps: 3746
Producer:1 finished, events: 200000, Time(s): 55, tps: 3635
Producer:2 finished, events: 200000, Time(s): 55, tps: 3634
Producer:4 finished, events: 200000, Time(s): 55, tps: 3629

执行看到队列进入flow模式(通过管理界面),但一切恢复正常。

我确实看到你的工人受到流量控制......

"pool-2-thread-3" prio=10 tid=0x00007f4af4849800 nid=0x65d5 runnable [0x00007f4ae082f000]
 java.lang.Thread.State: RUNNABLE
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)

你在兔子日志中看到了什么吗?您在管理界面上看到的关于消息率,状态等的内容是什么?

无论如何,这似乎与Spring AMQP没有任何关系;你需要联系rabbitmq-users谷歌小组的兔子家伙。

(我正在用rabbitmq 3.4.2测试)。

<强> EDIT2:

完全安装3.5.2 ......

Start producer with configuration [threadCount=5, events=200000, sleep=0]
Producer:0 finished, events: 200000, Time(s): 39, tps: 5091
Producer:1 finished, events: 200000, Time(s): 39, tps: 5002
Producer:2 finished, events: 200000, Time(s): 40, tps: 4954
Producer:3 finished, events: 200000, Time(s): 40, tps: 4951
Producer:4 finished, events: 200000, Time(s): 40, tps: 4939

我在管理界面中看到没有flow状态(在队列中,但是频道/连接显示它们在流中,但又一次恢复)。