尝试使用Celery

时间:2016-09-15 14:14:30

标签: python django sockets redis redis-cluster

我的环境: 我有三个Ubuntu服务器。 一台服务器用作使用Nginx的负载均衡器。 其他两个服务器包含完全相同的项目(除了redis之外,其中一个框是主框,另一个是从属框)。

我使用的程序/应用程序 蟒蛇, Gunicorn, Django的, 芹菜, Redis的, 前哨

我的项目的作用: 我有一个URL接收GET请求,做一些关于请求的逻辑并保存到redis。

def save(key, value):
    conn = redis_cluster_connect()
    print conn
    conn.set(key, value)
    conn.set('bar','foo')

当我的连接是:

时,这工作正常
def redis_connect():
    print 'redis connected'
    return redis.Redis(host="xxx.xx.xxx.x", port=6380, db=1) # CHANGE TO THE REDIS PORT THAT IS ACTIVE

但是当我使用群集连接时:

def redis_cluster_connect():
    startup_nodes = [{"host": "xxx.xx.xxx.x", "port": "6380"}]
    rc = StrictRedisCluster(startup_nodes=startup_nodes, decode_responses=True)

return rc

我得到的错误:

[2016-09-15 13:28:59,682: INFO/MainProcess] Received task: testapp.tasks.mobilise_stops[cc64c425-bd37-4896-b6ab-4319de5fb743]
    [2016-09-15 13:28:59,684: WARNING/Worker-1] Oh no! Task failed: RedisClusterException("ERROR sending 'cluster slots' command to redis server: {'host': 'xxx.xx.xxx.x', 'port': '6380'}",)
    [2016-09-15 13:28:59,685: ERROR/MainProcess] Task testapp.tasks.mobilise_stops[cc64c425-bd37-4896-b6ab-4319de5fb743] raised unexpected: RedisClusterException("ERROR sending 'cluster slots' command to redis server: {'host': 'xxx.xx.xxx.x', 'port': '6380'}",)
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 240, in trace_task
        R = retval = fun(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 438, in __protected_call__
        return self.run(*args, **kwargs)
      File "/var/www/gateway/venv/gateway/testapp/tasks.py", line 50, in mobilise_stops
        save(mobilise_stops.request.id, 'Success')
      File "/var/www/gateway/venv/gateway/testapp/tasks.py", line 28, in save
        conn = redis_cluster_connect()
      File "/var/www/gateway/venv/gateway/testapp/tasks.py", line 19, in redis_cluster_connect
        rc = StrictRedisCluster(startup_nodes=startup_nodes, decode_responses=True)
      File "/usr/local/lib/python2.7/dist-packages/rediscluster/client.py", line 157, in __init__
        **kwargs
      File "/usr/local/lib/python2.7/dist-packages/rediscluster/connection.py", line 83, in __init__
        self.nodes.initialize()
      File "/usr/local/lib/python2.7/dist-packages/rediscluster/nodemanager.py", line 148, in initialize
        raise RedisClusterException("ERROR sending 'cluster slots' command to redis server: {0}".format(node))
    RedisClusterException: ERROR sending 'cluster slots' command to redis server: {'host': 'xxx.xx.xxx.x', 'port': '6380'}

当我运行redis-cli并在主服务器上设置变量(在服务器1上)时,我可以在从服务器上检索它(在服务器2上)

server 1:
xxx.xx.xxx.x:6380> set test 100
OK

server 2:
xxx.xx.xxx.x:6381> get test 
"100"

当我尝试在redis客户端中执行cluster slots命令时,我得到与上面相同的错误。

xxx.xx.xxx.x:6380> cluster slots
(error) ERR unknown command 'cluster'

要考虑的一件事是我的redis配置文件没有tcp套接字,当我尝试使用tcp时,redis无效。当我尝试运行redis时,它给了我一个错误。这已更改为默认的redis.conf文件中的默认Unixsocket

自:

tcp-backlog 511

要:

unixsocket /var/run/redis/redis.sock
unixsocketperm 700

2 个答案:

答案 0 :(得分:0)

您没有运行redis群集,因此尝试仅运行redis-cluster命令将失败。

答案 1 :(得分:0)

对于我的情况,问题出现在版本redis-py-cluster(1.3.4)中,重新安装到版本1.3.1后,一切都适合我。我猜这个版本的python包存在一些问题。