我在AWS上托管的redis服务器上有几个Hashes（大约1700000个密钥），每个都有单个键值对。它们采用以下格式：
127.0.0.1:6379[10]> HGETALL "date:20170420:len:3"
1) "prices"
2) "{\"price\": \"500.84\", \"tax\": \"10.00\", \"total\": \"510.84\"}"
我有一个API，它从这些哈希集中读取值，并整理它们并给出响应。这是代码的样子：
def fetch_prices(dates):
import time
import math
st = time.time()
# Total keys: len(date) * len(l)
dates = {"date:{date}:len:{l}".format(date=d) for d in dates for l in [1, 2, 3]}
pipe = r.pipeline()
stc = time.time()
timer_logger.info("Total HGET operation: %s", len(dates))
for d in dates:
pipe.hget(d, 'prices')
etc = time.time()
timer_logger.info("for loop on hotel total time: %s ms", math.ceil((etc - stc) * (10 ** 5)) / (10 ** 2))
stc = time.time()
values = pipe.execute()
etc = time.time()
timer_logger.info("hgets total time: %s ms", math.ceil((etc - stc) * (10 ** 5)) / (10 ** 2))
stc = time.time()
return_value = [json.loads(v, object_hook=price_hook) for v in values if v]
return return_value
现在，当我使用Apache Bench以200并发方式访问此API时，我发现hget所花费的总时间正在逐渐增加：
ab -n 200 -c 200 "http://api.hitting.cache/"
这里是hget操作打印总时间的对数：
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 17.81 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 291.11 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 108.25 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 171.45 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 343.58 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 175.16 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 241.48 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 425.97 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 238.87 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 310.88 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 297.65 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 218.19 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 352.3 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 379.52 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 400.81 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 304.29 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 459.14 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 474.69 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 378.81 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 537.63 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 537.2 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 455.07 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 605.66 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 602.16 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 534.19 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 588.35 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 702.08 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 696.57 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 652.01 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 754.78 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 455.07 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 605.66 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 602.16 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 534.19 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 588.35 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 702.08 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 696.57 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 652.01 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 754.78 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 495.6 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 718.97 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 431.86 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 485.82 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 791.71 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 538.0 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 850.11 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 611.74 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 903.57 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 664.5 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 952.11 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 529.03 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 629.71 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 678.35 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 743.31 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 804.98 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 831.75 ms
[22/Apr/2017 07:05:07] INFO [timer.caching:106] hgets total time: 857.5 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 939.2 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1264.54 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 876.55 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 990.76 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1292.21 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1035.62 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1340.49 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1105.4 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1378.29 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1140.46 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1416.63 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1109.07 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1102.94 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1457.5 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 919.83 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1487.42 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 880.02 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1535.06 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 904.95 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1574.73 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 973.43 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1607.64 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 957.17 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1648.63 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1020.83 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1714.63 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1751.34 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1789.32 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1816.73 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1862.82 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1899.62 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1935.82 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 1971.54 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 2006.62 ms
[22/Apr/2017 07:05:08] INFO [timer.caching:106] hgets total time: 2036.07 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2074.11 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2112.46 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2153.43 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2186.09 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2225.63 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2263.8 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2297.19 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2344.4 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 2380.6 ms
[22/Apr/2017 07:05:09] INFO [timer.caching:106] hgets total time: 1087.45 ms
这些数字是每个API匹配的总 46-50 HGET 操作。 正如你可以看到下半部分，时间达到2秒，对于50次HGET操作看起来太多了。
我在Python版本2.7上测试了这个，redis-py == 2.10.5，hiredis-parser == 0.2.0。我和gevent worker一起在gunicorn服务器上运行它。
API托管在AWS t2.medium（2 CPU，8 GB内存）计算机上。并且AWS Elastic Cache上的cache.m4.large（2个CPU，6.4 GB内存）redis实例。
我正在另一台AWS计算机上运行apache bench测试，该计算机托管在同一可用区中。有什么建议？我在这里做错了吗？
这是其中一个API调用的分析器编号（这个让我感到惊讶）。在系统已经加载200并发时获取此数字：
980279 function calls (933994 primitive calls) in 5.772 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
45 0.000 0.000 123.777 2.751 /usr/local/lib/python2.7/site-packages/gevent/hub.py:534(switch)
45 0.005 0.000 123.776 2.751 {method 'switch' of 'greenlet.greenlet' objects}
45/1 0.001 0.000 5.839 5.839 /usr/local/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/hooks/framework_django.py:489(wrapper)
45/1 0.001 0.000 5.839 5.839 /usr/local/lib/python2.7/site-packages/django/views/decorators/csrf.py:57(wrapped_view)
44/1 0.001 0.000 5.830 5.830 /usr/local/lib/python2.7/site-packages/django/views/generic/base.py:64(view)
44/1 0.001 0.000 5.830 5.830 /usr/src/app/pricing/api/api.py:87(dispatch)
44/1 0.002 0.000 5.829 5.829 /usr/local/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/hooks/component_djangorestframework.py:4(_nr_wrapper_APIView_dispatch_)
44/1 0.006 0.000 5.825 5.825 /usr/local/lib/python2.7/site-packages/rest_framework/views.py:428(dispatch)
44/1 0.058 0.001 5.825 5.825 /usr/src/app/pricing/api/pricing_api.py:80(get)
44/1 0.008 0.000 5.769 5.769 /usr/src/app/timer/caching.py:169(fetch_prices)
44/1 0.007 0.000 5.769 5.769 /usr/src/app/timer/caching.py:82(fetch_prices)
44/1 0.000 0.000 5.750 5.750 /usr/local/lib/python2.7/site-packages/redis/client.py:2605(execute)
44/1 0.019 0.000 5.750 5.750 /usr/local/lib/python2.7/site-packages/redis/client.py:2492(_execute_transaction)
2112/48 0.018 0.000 5.750 0.120 /usr/local/lib/python2.7/site-packages/redis/client.py:2582(parse_response)
2112/48 0.004 0.000 5.750 0.120 /usr/local/lib/python2.7/site-packages/redis/client.py:583(parse_response)
2112/48 0.003 0.000 5.750 0.120 /usr/local/lib/python2.7/site-packages/redis/connection.py:574(read_response)
2112/48 0.012 0.000 5.750 0.120 /usr/local/lib/python2.7/site-packages/redis/connection.py:329(read_response)
44/1 0.001 0.000 5.750 5.750 /usr/local/lib/python2.7/site-packages/gevent/socket.py:420(recv_into)
43/1 0.000 0.000 5.750 5.750 /usr/local/lib/python2.7/site-packages/gevent/socket.py:286(_wait)
46/1 0.002 0.000 5.750 5.750 /usr/local/lib/python2.7/site-packages/gevent/hub.py:343(wait)
43/1 0.001 0.000 5.749 5.749 /usr/local/lib/python2.7/site-packages/gevent/greenlet.py:320(run)
43/1 0.022 0.001 5.749 5.749 /usr/local/lib/python2.7/site-packages/gunicorn/workers/async.py:30(handle)
43/1 0.000 0.000 5.735 5.735 /usr/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py:157(handle_request)
43/1 0.009 0.000 5.735 5.735 /usr/local/lib/python2.7/site-packages/gunicorn/workers/async.py:87(handle_request)
86/2 0.004 0.000 5.731 2.866 /usr/local/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/api/web_transaction.py:1172(_nr_wsgi_application_wrapper_)
86/2 0.001 0.000 5.729 2.864 /usr/local/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/api/web_transaction.py:703(__iter__)
86/2 0.002 0.000 5.729 2.864 /usr/local/lib/python2.7/site-packages/newrelic-2.50.0.39/newrelic/api/web_transaction.py:1079(__call__)
43/1 0.001 0.000 5.722 5.722 /usr/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py:162(__call__)
43/1 0.008 0.000 5.722 5.722 /usr/local/lib/python2.7/site-packages/django/core/handlers/base.py:94(get_response)
745 0.008 0.000 2.930 0.004 /usr/local/lib/python2.7/logging/__init__.py:1267(_log)
659 0.003 0.000 2.902 0.004 /usr/local/lib/python2.7/logging/__init__.py:1157(info)
745 0.002 0.000 2.803 0.004 /usr/local/lib/python2.7/logging/__init__.py:1288(handle)
745 0.006 0.000 2.797 0.004 /usr/local/lib/python2.7/logging/__init__.py:1320(callHandlers)
2593 0.017 0.000 2.792 0.001 /usr/local/lib/python2.7/logging/__init__.py:746(handle)
2593 0.037 0.000 2.694 0.001 /usr/local/lib/python2.7/logging/__init__.py:849(emit)
2593 2.446 0.001 2.446 0.001 {method 'write' of 'file' objects}
43 0.000 0.000 0.588 0.014 /usr/local/lib/python2.7/site-packages/django/template/response.py:149(render)
43 0.000 0.000 0.587 0.014 /usr/local/lib/python2.7/site-packages/rest_framework/response.py:39(rendered_content)
43 0.359 0.008 0.585 0.014 /usr/src/app/pricing/api/api.py:21(render)
是gevent的罪魁祸首吗？那些{方法＆＃39;写＆＃39; ＆＃39; file＆＃39;对象}可能会持续2.5秒？
我已经看到了延迟，看起来不错：
$ redis-cli -h host -p 6379 --latency
min: 0, max: 10, avg: 0.67 (23382 samples)