Question

我注意到我的系统工作缓慢取决于在生产系统上获取芹菜异步任务结果。在调试期间，我发现了redis-py基本操作的巨大延迟。

Linux 3.16.0-36-generic Ubuntu 14.04
Python 2.7.6
redis-py 2.10.3
Redis 2.8.19

import redis

r = redis.Redis()
r.set('foo', 'bar')  # takes about 2 minutes
r.set('bar', 'foo')  # runs 'instantly'

我确定这不是redis或连接问题（redis在本地运行）

redis-benchmark -t set -n 10000 q
===== q ======
  10000 requests completed in 0.17 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

99.01% <= 1 milliseconds
99.11% <= 2 milliseconds
99.51% <= 15 milliseconds
100.00% <= 15 milliseconds
57471.27 requests per second

然后我尝试对其进行配置并获得下一个cProfile输出：

   23    0.000    0.000    0.000    0.000 {len}
    1    0.000    0.000    0.000    0.000 connection.py:112(_read_from_socket)
    1    0.000    0.000    0.000    0.000 connection.py:156(readline)
    1    0.000    0.000    0.000    0.000 connection.py:175(purge)
    1    0.000    0.000    0.000    0.000 connection.py:192(__init__)
    1    0.000    0.000    0.000    0.000 connection.py:203(on_connect)
    1    0.000    0.000    0.000    0.000 connection.py:223(read_response)
    1    0.000    0.000    0.000    0.000 connection.py:385(__init__)
    1    0.000    0.000  127.266  127.266 connection.py:428(connect)
    1    0.000    0.000  127.266  127.266 connection.py:451(_connect)
    1    0.000    0.000    0.000    0.000 connection.py:501(on_connect)
    1    0.000    0.000  127.266  127.266 connection.py:529(send_packed_command)
    1    0.000    0.000  127.266  127.266 connection.py:554(send_command)
    1    0.000    0.000    0.000    0.000 connection.py:566(read_response)
    3    0.000    0.000    0.000    0.000 connection.py:577(encode)
    1    0.000    0.000    0.000    0.000 connection.py:593(pack_command)
    1    0.000    0.000    0.000    0.000 connection.py:68(__init__)
    2    0.000    0.000    0.000    0.000 connection.py:858(_checkpid)
    1    0.000    0.000    0.000    0.000 connection.py:868(get_connection)
    1    0.000    0.000    0.000    0.000 connection.py:878(make_connection)
    1    0.000    0.000    0.000    0.000 connection.py:885(release)
    1    0.000    0.000    0.000    0.000 connection.py:99(__init__)
    2    0.000    0.000    0.000    0.000 socket.py:185(__init__)
    1    0.000    0.000    0.000    0.000 socket.py:192(close)
    8    0.000    0.000  127.264   15.908 socket.py:223(meth)
    1    0.001    0.001    0.001    0.001 {_socket.getaddrinfo}
    1    0.000    0.000    0.000    0.000 {cStringIO.StringIO}
   20    0.000    0.000    0.000    0.000 {getattr}
   11    0.000    0.000    0.000    0.000 {isinstance}
   13    0.000    0.000    0.000    0.000 {len}
    1    0.000    0.000    0.000    0.000 {method 'add' of 'set' objects}
    2    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
    2  127.264   63.632  127.264   63.632 {method 'connect' of '_socket.socket' objects}
    1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    2    0.000    0.000    0.000    0.000 {method 'endswith' of 'str' objects}
    4    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
    1    0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
    2    0.000    0.000    0.000    0.000 {method 'readline' of 'cStringIO.StringO' objects}
    1    0.000    0.000    0.000    0.000 {method 'recv' of '_socket.socket' objects}
    1    0.000    0.000    0.000    0.000 {method 'remove' of 'set' objects}
    4    0.000    0.000    0.000    0.000 {method 'seek' of 'cStringIO.StringO' objects}
    1    0.000    0.000    0.000    0.000 {method 'sendall' of '_socket.socket' objects}
    2    0.000    0.000    0.000    0.000 {method 'setsockopt' of '_socket.socket' objects}
    3    0.000    0.000    0.000    0.000 {method 'settimeout' of '_socket.socket' objects}
    1    0.000    0.000    0.000    0.000 {method 'truncate' of 'cStringIO.StringO' objects}
    1    0.000    0.000    0.000    0.000 {method 'write' of 'cStringIO.StringO' objects}
    3    0.000    0.000    0.000    0.000 {posix.getpid}
   18    0.000    0.000    0.000    0.000 {setattr}

为什么redis-py打开连接很长时间？

UPD：

通过socket_connect_timeout连接打开持续时间〜超时。没有引发异常，r.set('foo', 'bar')返回True。

UPD：

>>> socket.getaddrinfo('localhost', 6379)
[(10, 1, 6, '', ('::1', 6379, 0, 0)), (10, 2, 17, '', ('::1', 6379, 0, 0)), (10, 3, 0, '', ('::1', 6379, 0, 0)), (2, 1, 6, '', ('127.0.0.1', 6379)), (2, 2, 17, '', ('127.0.0.1', 6379)), (2, 3, 0, '', ('127.0.0.1', 6379))]

>>> sock_info = socket.getaddrinfo('localhost', 6379)
>>> sock = socket.socket(*sock_info[0][:3])
>>> sock.connect(('localhost', 6379))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/socket.py", line 224, in meth
  return getattr(self._sock,name)(*args)
  socket.error: [Errno 110] Connection timed out
>>> sock = socket.socket(*sock_info[1][:3])
>>> sock.connect(('localhost', 6379))  # success

Redis-py基本操作延迟很大

0 个答案: