How do I implement a Redis equivalent of a "unique index" while avoiding race conditions?

Question

I'm experimenting with using Redis as persistent storage. I want to store users in Redis, but I want the user ID as well as the user email address to be "unique".

Here's where I'm at:

MULTI
SET users:1 "<user encoded as JSON>"
HSET users-indexes:email user-email@gmail.com 1
EXEC

I'm using transactions because I don't want the database to end up in an invalid state if Redis crashes.

Currently this will simply silently overwrite existing values. I need the transaction to fail/roll-back if the email address already exists in the hash.

I could use HSETNX to prevent the index from being overwritten, but the user object would already be overwritten at that point. I could check if the email address exists in the index in advance, but then I'd have a race condition with other clients.

I could add a write lock to my application, which would solve my problem as long as the only Redis client is a single instance of my application, otherwise there would still be a race condition.

A pretty perfect solution in my scenario would be locking the Redis database temporarily while users are being created. Users are created so rarely that it wouldn't be a significant performance impact. That doesn't seem to be possible though.

Am I missing a simple solution?

Answer 1

One approach could be:

• A SET (user:email) to store emails
• A HASH (user:data)to add user objects indexed by user id

Before to implement the transaction (EXEC), from the application, you run SADD user:email user-email command, if it returns 0, the email already existed before, so you stop saving the user. If it returns 1, user didn't exist, so you run HGET user:data user-id command, if it return NULL, user didn't exists so you can execute the transaction, otherwise you stop saving the current user, you then run:

MULTI
SADD user:email user-email
HSET user:data user-id "<user encoded as JSON>"
EXEC

If you really keep a SET for every user, you only have to change the HASH which contains all the users to a SET for every user. But I recommend you to use a HASH indexed by user-id so you keep all your users in one Redis structure and Redis will manage fewer keys when you have to find an user.

Anyway and basically, the idea here is to use previous queries to know if the email already exists.

One of the main reason to use previous queries is because Transactions in Redis (EXEC) are not similar to the transaction as we know for databases, there is no roll-back, if one command inside of a transaction fails there is no roll-back, the transaction continues with the next command until it finishes to process all commands. The good news is that transaction returns a kinda of array of all returned values for each command.

Here a clear article about why is not possible to roll back a transaction on Redis: http://openmymind.net/You-Cant-Rollback-Redis-Transaction/

I suggest you read the Transaction topic in Redis documentation https://redis.io/topics/transactions to understand how Redis return data after running a transaction

Answer 2

您不需要屏蔽您的应用或Redis，您可以将计数器用作唯一ID生成器和一组电子邮件地址。

如果SADD global_emails user@email返回1，INCR global_id将为您提供下一个唯一ID，存储他的信息（在HASH中）并返回成功。 否则，该电子邮件已被使用。

您也可以将此过程编写为Lua，将SCRIPT LOAD写入Redis以减少网络延迟。