RabbitMQ随机延迟我的邮件

Question

这是我的rabbitmqctl status：

[{pid,32074},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.2.2"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.2.2"},
      {rabbit,"RabbitMQ","3.2.2"},
      {os_mon,"CPO  CXC 138 46","2.2.7"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.2.2"},
      {webmachine,"webmachine","1.10.3-rmq3.2.2-gite9359c7"},
      {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.2.2-git680dba8"},
      {xmerl,"XML parser","1.2.10"},
      {inets,"INETS  CXC 138 49","5.7.1"},
      {mnesia,"MNESIA  CXC 138 12","4.5"},
      {amqp_client,"RabbitMQ AMQP Client","3.2.2"},
      {sasl,"SASL  CXC 138 11","2.1.10"},
      {stdlib,"ERTS  CXC 138 10","1.17.5"},
      {kernel,"ERTS  CXC 138 10","2.14.5"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:32:32] [rq:32] [async-threads:30] [kernel-poll:true]\n"},
 {memory,
     [{total,1954308048},
      {connection_procs,619048024},
      {queue_procs,166111144},
      {plugins,4423520},
      {other_proc,46207032},
      {mnesia,44407568},
      {mgmt_db,331614464},
      {msg_index,6694584},
      {other_ets,30005328},
      {binary,63825504},
      {code,17629100},
      {atom,6531121},
      {other_system,617810659}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,53967541043},
 {disk_free_limit,50000000},
 {disk_free,51883839488},
 {file_descriptors,
     [{total_limit,655260},
      {total_used,12659},
      {sockets_limit,589732},
      {sockets_used,12657}]},
 {processes,[{limit,1048576},{used,125740}]},
 {run_queue,1},
 {uptime,33320350}]

我的队列大部分时间都是空的。但我的RabbitMQ中有84440个交换和8917个队列。

我的问题是，自我发送邮件以来，邮件可能会延迟很长时间。

我试图从现场获取一些信息。这是延迟发生时我的监视器图表。 我们可以看到的是概述和我的交流很好。但我的消息在队列中是延迟交付和确认。

但是从我的应用日志开始。

2017-08-10 17:23:08.738 4219 INFO trove.openstack.common.rpc.amqp [-] [ProxyCallback]Received Message with Timestamp:2017-08-10T17:23:08, duration:0.737949s, unique_id:a17186068cae447bbada7a0f24ff45ef

收到了17:23消息，没有延迟。然后我将ACK发送回MQ：

2017-08-10 17:23:08.739 4219 DEBUG trove.openstack.common.rpc.common [-] Consume Massage with ACK True

当我的17:43消息被延迟了215.895117s时，真的很困惑我。

2017-08-10 17:43:53.895 4219 INFO trove.openstack.common.rpc.amqp [-] [ProxyCallback]Received Message with Timestamp:2017-08-10T17:40:18, duration:215.895117s, unique_id:dc04b94c8fa64978bc9d681b020f4500

Answer 1

最后，我发现这是我的网络问题。 有时消息将在交付中丢失，而RabbitMQ需要很长时间的超时并重试消息传递。 结果我的消息延迟了。

问题源于安全组规则，在900s没有数据包发送后，网关将断开tcp连接而不重置（只丢弃数据包）。

系统范围的tcp保持活跃状态​​（tcp_keepalive_intvl）设置为1200秒。

所以我只需将tcp_keepalive_intvl设置为sysctl到600秒，问题就解决了。