Question

我有一个3节点EC2 redis群集设置，我正在尝试使用管道模式将记录添加到redis（使用sadd）。

在3个节点中添加约70/82和81个密钥后出现以下错误：

    Exception in thread "main" redis.clients.jedis.exceptions.JedisMovedDataException: MOVED 1539 172.31.59.103:6379
at redis.clients.jedis.Protocol.processError(Protocol.java:93)
at redis.clients.jedis.Protocol.process(Protocol.java:122)
at redis.clients.jedis.Protocol.read(Protocol.java:191)
at redis.clients.jedis.Connection.getOne(Connection.java:258)
at redis.clients.jedis.ShardedJedisPipeline.sync(ShardedJedisPipeline.java:44)
at org.hu.e63.MovieLens21MPipeline.push(MovieLens21MPipeline.java:47)
at org.hu.e63.MovieLens21MPipeline.main(MovieLens21MPipeline.java:53

我看过this帖子，我的代码看起来很像：

输入文件来自此处（ratings.csv）：http://files.grouplens.org/datasets/movielens/ml-latest-small.zip

以下是代码：

import java.io.File;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;

import redis.clients.jedis.JedisShardInfo;
import redis.clients.jedis.ShardedJedis;
import redis.clients.jedis.ShardedJedisPipeline;

public class MovieLens21MPipeline {
    ShardedJedis jedis;

    public MovieLens21MPipeline() {
        JedisShardInfo si = new JedisShardInfo("172.31.59.103", 6379, 5000);
        List<JedisShardInfo> list = new ArrayList<JedisShardInfo>();
        list.add(si);
        list.add(new JedisShardInfo("172.31.59.104", 6379, 5000));
        list.add(new JedisShardInfo("172.31.59.105", 6379, 5000));

        jedis = new ShardedJedis(list);
    }

    public void push() {
        ShardedJedisPipeline pipeline = jedis.pipelined();
        Scanner s;

        try {
            s = new Scanner(new File("input/ratings.csv"));

            StringBuilder key = new StringBuilder();
            String s1 = s.nextLine(); // Skip first line
            while (s.hasNextLine()) {
                s1 = s.nextLine();
                String[] spl = s1.split(",");
                // key="u:"+spl[0]+":m";
                key.append("u:").append(spl[0]).append(":m");
                pipeline.sadd(key.toString(), spl[1]);
                key.setLength(0);
            }
        } catch (Exception e) {
            // TODO Auto-generated catch block
            System.out.println("bye" + e.getMessage());
        }

        pipeline.sync();
    }

    public static void main(String[] args) {
        MovieLens21MPipeline obj = new MovieLens21MPipeline();
        long startTime = System.currentTimeMillis();
        obj.push();
        long endTime = System.currentTimeMillis();
        double d = 0.0;
        d = (double) (endTime - startTime);
        System.out.println("Throughput: " + d);
    }
}

任何帮助表示赞赏。谢谢

Answer 1

MOVED表示您正在使用Redis群集。 ShardedJedis不适用于Redis群集，因此您应该使用JedisCluster。请注意，JedisCluster没有管道模式，因此您可能希望逐个发送操作。

希望这有帮助。

Answer 2

就像Lim所说的那样，您正在集群模式下使用Redis。使用JedisCluster可以解决此问题。另外，作为补充，为什么JedisCluster可以工作？

我们从JedisCluster源代码中知道这一点。

private T runWithRetries(byte[] key, int attempts, boolean tryRandomNode, boolean asking) {
    if (attempts <= 0) {
        throw new JedisClusterMaxRedirectionsException("Too many Cluster redirections?");
    }

    Jedis connection = null;
    try {
        // omit ... get connection
        return execute(connection);

    } catch (JedisNoReachableClusterNodeException jnrcne) {
        throw jnrcne;
    } catch (JedisConnectionException jce) {
        // release current connection before recursion
        // omit

        return runWithRetries(key, attempts - 1, tryRandomNode, asking);
    } catch (JedisRedirectionException jre) {
        // if MOVED redirection occurred,
        if (jre instanceof JedisMovedDataException) {
            // here is the key point 
            // it rebuilds cluster's slot cache
            // recommended by Redis cluster specification
            // Jedis or ShardedJedis won't run this code
            this.connectionHandler.renewSlotCache(connection);
        }

        // omit ... 

        return runWithRetries(key, attempts - 1, false, asking);
    } finally {
        releaseConnection(connection);
    }
}

JedisMovedDataException（当使用带有管道的sadd时）

2 个答案: