Question

当我向Redis添加一百万（1,000,000）时，没关系当我添加两百万（2,000,000）条记录时，我收到错误Connection reset by peer: socket write error;

列表的最大长度为2 ³² - 1个元素（4294967295，每个列表超过40亿个元素）。

/*Creating the json list*/
Gson gson = new GsonBuilder().create();
List<String> employeeList = new ArrayList<String>();
for (int i = 1; i <= 2000000; i++) {
    Employee employee = new Employee(i + "", "Jhon", "My Country", "redis@gmail.com", "+777 92157325");
    String json = gson.toJson(employee);
    employeeList.add(json);
}

/*add json list to Redis*/
Jedis jedis = pool.getResource();
// employeeList size is = 2,000,000
String[] jsonArray = employeeList.toArray(new String[employeeList.size()]);
jedis.lpush("employee_list_1", jsonArray);

日志

Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Connection reset by peer: socket write error
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:83)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:63)
    at redis.clients.jedis.Connection.sendCommand(Connection.java:84)
    at redis.clients.jedis.BinaryClient.lpush(BinaryClient.java:281)
    at redis.clients.jedis.Client.lpush(Client.java:205)
    at redis.clients.jedis.Jedis.lpush(Jedis.java:869)
    at com.mutu.redis.AddJosn.add(AddJosn.java:29)
    at com.mutu.redis.AddJosn.main(AddJosn.java:48)
Caused by: java.net.SocketException: Connection reset by peer: socket write error
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
    at redis.clients.util.RedisOutputStream.flushBuffer(RedisOutputStream.java:31)
    at redis.clients.util.RedisOutputStream.write(RedisOutputStream.java:54)
    at redis.clients.util.RedisOutputStream.write(RedisOutputStream.java:44)
    at redis.clients.jedis.Protocol.sendCommand(Protocol.java:79)
    ... 7 more

如何通过单个事务/流程添加两个或更多百万条记录？

Answer 1

通过对等方重置连接意味着Redis断开Jedis实例下面的套接字连接。检查Redis日志中记录了哪条警告/错误消息。

顺便说一句，因为lpush with array被发送到一个命令，请求体应该非常庞大。我认为你只是出于测试目的这样做，但我建议你需要使用multi-exec并将大量数组中的请求分解为大量请求。它仍然保证原子性。

如果您不需要具有原子性，则可以使用管道来分解您的请求。

通过Jedis lpush将百万条记录添加到Redis中 - 通过对等方重置连接：套接字写入错误

1 个答案: