我正在尝试创建某种队列,许多进程可以访问这些队列并以原子方式处理项目:
while True:
s= db.queue.find_and_modify({'service':'lock'},{'$set':{'locked':True}},upsert=True,new=False) # serves as a lock over the whole collection
if s and s['locked']: # already locked by another process
time.sleep(random.randint(2,10)) # sleep and try again
continue
else: # the lock was put by this process (atomic find_and_modify)
db.queue.update({'service':'lock'},{'$set':{'pid':process_id}}, upsert=True) # register the process who put the lock
res = db.queue.find({'when':{'$lte':last},'locked':False}).limit(1).sort('when',1) #select an available item
to_return = None #initialize with None
if res.count(): # 1 or more
to_return = res[0] #get the first item
to_return['locked'] = True # lock it
to_return['pid'] = process_id # register the process locking it
db.queue.save(to_return)
db.queue.update({'service':'lock'},{'$set':{'locked':False}}) # remove the global lock
return to_return
有时候我会看到一个进程DEFUNCT 在日志中我看到了例外:
Traceback (most recent call last):
File "driver.py", line 22, in getUserFromQueue
to_return = res[0]
IndexError("no such item for Cursor instance")
这意味着光标res
为空,
但是由于存在全局锁定,它怎么可能是空的:只有一个进程查询项目,它已经通过了条件if res.count()
,即。光标中至少有一个项目
我必须创建全局锁的原因是我不能使用限制并使用find_and_modify进行排序