使用Redis实现可靠队列的一般方法是使用RPOPLPUSH-从“等待”队列弹出,再推送到“处理”队列。然后,工作人员处理作业,并将其从“处理”队列中删除。 如果工作对象无法将其从“处理”队列中删除,则另一个监视过程会将作业从“处理”队列中推回到“等待”队列中。 此处的详细信息:https://redis.io/commands/rpoplpush(模式:可靠队列)
这是实施Amazon SQS的方法之一,例如“可见性超时”(https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html)。
或者,我有以下方案,不需要额外的“监视”过程:
请注意,这使用此处描述的redis锁定的一般模式:
https://redis.io/topics/distlock
1. Jobs are queued to a "waiting" queue
2. For a worker to be able to process a job it needs to lock it
3. Locking is implemented using "SET NX" command
4. Hence to work on a job, the worker needs to:
index := 0
while true {
results := lrange waiting_queue index index
if results is nil or empty
// nothing to process, break out of loop
break;
else
// try and lock it
SET job.Id my_random_value NX PX 30000
If "SET" is successful, then lock acquired on Job {
// mark it so that we can delete it later
job.mark = my_random_value
jobJson := toJson(job)
LSET waiting_queue index jobJson
process the job
Del the lock Del(jobId, my_random_value) (Delete only if value matches)
lrem job 1 jobJson
} else {
index++
}
}
对此方法有何想法?