我正在我的Ubuntu工作站上运行一些测试。这些基准测试从填充队列开始,该队列运行速度非常慢:
import pika
import datetime
if __name__ == '__main__':
try:
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='hello_durable', durable=True)
started_at = datetime.datetime.now()
properties = pika.BasicProperties(delivery_mode=2)
for i in range(0, 100000):
channel.basic_publish(exchange='',
routing_key='hello',
body='Hello World!',
properties=properties)
if i%10000 == 0:
duration = datetime.datetime.now() - started_at
print(i, duration.total_seconds())
print(" [x] Sent 'Hello World!'")
connection.close()
now = datetime.datetime.now()
duration = now - started_at
print(duration.total_seconds())
except Exception as e:
print(e)
发送10K消息需要30多秒。根据top命令,工作站有12个核心,它们并不繁忙。有超过8Gb的可用内存。队列是否持久并不重要。
我们如何加快发送邮件的速度?
答案 0 :(得分:2)
从BlockingConnection切换到SelectConnection产生了巨大的差异,加快了这个过程近五十倍。我需要做的就是修改the following tutorial:中的示例,在循环中发布消息:
import pika
# Step #3
def on_open(connection):
connection.channel(on_channel_open)
# Step #4
def on_channel_open(channel):
channel.basic_publish('test_exchange',
'test_routing_key',
'message body value',
pika.BasicProperties(content_type='text/plain',
delivery_mode=1))
connection.close()
# Step #1: Connect to RabbitMQ
parameters = pika.URLParameters('amqp://guest:guest@localhost:5672/%2F')
connection = pika.SelectConnection(parameters=parameters,
on_open_callback=on_open)
try:
# Step #2 - Block on the IOLoop
connection.ioloop.start()
# Catch a Keyboard Interrupt to make sure that the connection is closed cleanly
except KeyboardInterrupt:
# Gracefully close the connection
connection.close()
# Start the IOLoop again so Pika can communicate, it will stop on its own when the connection is closed
connection.ioloop.start()
答案 1 :(得分:1)
假设您没有运行任何消费者These benchmarks start with populating a queue
。
由于您只发布消息,因此rabbitmq切换到流状态。更准确地说,您的交换和/或队列将进入流状态。
引自rabbitmq blog
这(大致)意味着客户受到速率限制;它会 喜欢发布得更快,但服务器无法跟上
我敢肯定,如果你看得足够近,你会看到消息的第一部分(在初始设置时,空队列)变快,但发送速率在某些时候急剧下降。