如何使用JAVA在Redis中进行大量插入？

Question

您好我需要多次插入表格

SADD键值

我有键值对，需要知道如何使用JAVA执行大量插入。我在Redis协议中编写了一个文件。如何进一步

Answer 1

如果您有输入写入Redis协议格式的输入，为什么不使用redis-cli或nc的管道模式？这是从http://redis.io/topics/mass-insert解释的。

如果您有质量（键，值）输入，那么您可以使用Jedis执行带有流水线操作的sadd以获得更高的性能。

下面的例子假设iter（Iterator）有元素，每个项目都是关键的“\ t”值格式。

try (Jedis jedis = new Jedis(host, port)) {
  Pipeline pipeline = jedis.pipelined();
  while (iter.hasNext()) {
    String[] keyValue = iter.next().split("\t");
    pipeline.sadd(keyValue[0], keyValue[1]);
    // you can call pipeline.sync() and start new pipeline here if you think there're so much operations in one pipeline
  }
  pipeline.sync();
}

Answer 2

如果您通过 Spring CacheManager 进行实际的读/写操作，并将 RedisTemplate 配置为使用 Redis 作为缓存，您也可以使用 executePipelined< /em> RedisTemplate 的方法，它将回调作为参数。回调需要定义 doInRedis 方法，该方法在 Redis 中执行您想要批量执行的工作（读/写操作）。

以下代码显示了通过调用 redisTemplate.opsForHash().put() 插入包装在具有 getKey() 和 getValue() 的 CacheableObject 接口中的对象列表。

@Component
public class RedisClient {
  @Autowired
  RedisTemplate redisTemplate;  
  
  //batch-insert using Redis pipeline, a list of objects into the cache specified by cacheName
  public void put(String cacheName, List<CacheableObject> objects) {
    try {
      this.redisTemplate.executePipelined(new RedisCallback<Object>() {
        @Override
        public Object doInRedis(RedisConnection connection) throws DataAccessException {
          for(CacheableObject object: objects) {
            redisTemplate.opsForHash().put(cacheName, object.getKey(), object.getValue()); 
          }
          return null;
        }
      });
    }
    catch(Exception e) {
        log.error("Error inserting objects into Redis cache: {}", e.getMessage());
    }
}

RedisTemplate 本身使用如下配置类进行配置：

@Configuration
@EnableCaching
public class RedisCacheConfig extends CachingConfigurerSupport implements 
                                                               CachingConfigurer {

  @Value("${redis.hostname}")
  private String redisHost;
  @Value("${redis.port}")
  private int redisPort;
  @Value("${redis.timeout.secs:1}")
  private int redisTimeoutInSecs;
  @Value("${redis.socket.timeout.secs:1}")
  private int redisSocketTimeoutInSecs;
  @Value("${redis.ttl.hours:1}")
  private int redisDataTTL;

  @Bean
  JedisConnectionFactory jedisConnectionFactory() {
     RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration(redisHost, redisPort);
     return new JedisConnectionFactory(redisStandaloneConfiguration);
  }
 
  @Bean
  public RedisTemplate<Object, Object> redisTemplate() {
    RedisTemplate<Object, Object> redisTemplate = new RedisTemplate<Object, Object>();
    redisTemplate.setConnectionFactory(jedisConnectionFactory());
    return redisTemplate;
}   

  @Bean
  public RedisCacheManager redisCacheManager (JedisConnectionFactory jedisConnectionFactory) {
    RedisCacheConfiguration redisCacheConfiguration = 
            RedisCacheConfiguration.defaultCacheConfig().disableCachingNullValues()
            .entryTtl(Duration.ofHours(redisDataTTL)) .serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(RedisSerializer.java()));
    redisCacheConfiguration.usePrefix();
    RedisCacheManager redisCacheManager = 
            RedisCacheManager.RedisCacheManagerBuilder.
            fromConnectionFactory(jedisConnectionFactory)
            .cacheDefaults(redisCacheConfiguration).build();
    redisCacheManager.setTransactionAware(true);
    return redisCacheManager;
}

  @Bean
  public JedisPoolConfig poolConfig() {
    final JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
    jedisPoolConfig.setTestOnBorrow(true);
    jedisPoolConfig.setMaxTotal(100);
    jedisPoolConfig.setMaxIdle(100);
    jedisPoolConfig.setMinIdle(10);
    jedisPoolConfig.setTestOnReturn(true);
    jedisPoolConfig.setTestWhileIdle(true);
    return jedisPoolConfig;
}

  @Override
  public CacheErrorHandler errorHandler() {
    return new RedisCacheErrorHandler();
  }
}