同步阻止多个资源

时间:2015-11-28 07:59:02

标签: python python-3.x python-asyncio

抽象情况。我们有两只羊,我们可以在时间上异步使用(信号量(2))和我们可以在时间使用的1门。我们想要通过门2次绵羊(每次我们需要1只绵羊和1个门,它持续1秒)并喂羊1次(需要1只绵羊和2秒)。这是代码示例:

acquire gate (spend 1)
acquire sheep (spend 1)
acquire sheep (spend 2) <-----
Spend sheep through a gate
release sheep (spend 1)
release gate (spend 1)
acquire sheep (feed 1)
acquire gate (spend 2)
Spend sheep through a gate
release sheep (spend 2)
release gate (spend 2)
Feed sheep
release sheep (feed 1)
[Finished in 3.2s]

输出:

spend 2

问题是程序不能以最佳方式工作,输出行№3中的原因:spend 1阻塞绵羊,而它不能立即使用它,它应该等待{{{{{{ 1}}。可以在spend 1时喂食的第二只可用绵羊浪费时间浪费:

enter image description here

程序应该如何工作的最佳方式:spend 1阻挡1只羊和1个门,spend 2看到它被阻挡的门,没有理由立即阻挡第二只羊。 feed 1可以阻止第二只绵羊并在spend 1运行时运行。在这种情况下,程序将在2秒而不是3秒内完成:

enter image description here

很容易看出你是否改变了主要聚集内的顺序。

资源应该被阻止,不仅是并行,而且同步,只有当羊可用且门可用时才应阻止绵羊和门。这样的事情:

while sheep.locked() or gate.locked():
    asyncio.sleep(0)
await asyncio.gather(
    sheep.acquire(), 
    gate.acquire()
)

但它看起来不像普遍而且很好的解决方案。可能是任何模式还是只是更好的方法来解决这个问题?欢迎任何想法。

2 个答案:

答案 0 :(得分:1)

您可以实现处理多个锁的asynchronous context manager。此对象应确保在等待另一个不可用的锁时它没有任何锁定:

class multilock(asyncio.locks._ContextManagerMixin):

    def __init__(self, *locks):
        self.released = list(locks)
        self.acquired = []

    async def acquire(self):
        while self.released:
            lock = self.released.pop()
            if lock.locked():
                self.release()
            await lock.acquire()
            self.acquired.append(lock)

    def release(self):
        while self.acquired:
            lock = self.acquired.pop()
            lock.release()
            self.released.append(lock)

示例:

async def test(lock1, lock2):
    async with multilock(lock1, lock2):
        print('Do something')

答案 1 :(得分:0)

基于this solution我为此示例创建了解决方案。我们需要两件事:

  1. locked()功能添加到SheepGate,检查是否 对象现在可以获得

  2. 添加并使用新的MultiAcquire任务,如果可以立即获取所有对象 >(否则暂停释放事件)

  3. 此处的最终代码,请参阅MultiAcquire - 它的主要内容:

    import asyncio
    
    
    class Sheep:
        _sem = asyncio.Semaphore(2)  # we have 2 avaliable sheeps at time
    
        def __init__(self, reason):
            self._reason = reason
    
        async def acquire(self):
            await type(self)._sem.acquire()
            print('acquire sheep ({})'.format(self._reason))
    
        def release(self):
            print('release sheep ({})'.format(self._reason))
            type(self)._sem.release()
    
        def locked(self):
            return type(self)._sem.locked()
    
    
    class Gate:
        _sem = asyncio.Semaphore(1)  # we have 1 avaliable gate at time
    
        def __init__(self, reason):
            self._reason = reason
    
        async def acquire(self):
            await type(self)._sem.acquire()
            print('acquire gate ({})'.format(self._reason))
    
        def release(self):
            print('release gate ({})'.format(self._reason))
            type(self)._sem.release()
    
        def locked(self):
            return type(self)._sem.locked()
    
    
    class MultiAcquire(asyncio.Task):
        _check_lock = asyncio.Lock()  # to suspend for creating task that acquires objects
        _release_event = asyncio.Event()  # to suspend for any object was released
    
        def __init__(self, locks):
            super().__init__(self._task_coro())
            self._locks = locks
            # Here we use decorator to subscribe all release() calls,
            # _release_event would be set in this case:
            for l in self._locks:
                l.release = self._notify(l.release)
    
        async def _task_coro(self):
            while True:
                # Create task to acquire all locks and break on success:
                async with type(self)._check_lock:
                    if not any(l.locked() for l in self._locks):  # task would be created only if all objects can be acquired
                        task = asyncio.gather(*[l.acquire() for l in self._locks])  # create task to acquire all objects 
                        await asyncio.sleep(0)  # start task without waiting for it
                        break
                # Wait for any release() to try again:
                await type(self)._release_event.wait()
            # Wait for task:
            return await task
    
        def _notify(self, func):
            def wrapper(*args, **kwargs):
                type(self)._release_event.set()
                type(self)._release_event.clear()
                return func(*args, **kwargs)
            return wrapper
    
    
    async def spend(reason):
        sheep = Sheep(reason)
        gate = Gate(reason)
        await MultiAcquire([sheep, gate])  # block 1 sheep, 1 gate
        await asyncio.sleep(1)  # 1 second
        print('Spend sheep through a gate')
        sheep.release()
        gate.release()
    
    
    async def feed(reason):
        sheep = Sheep(reason)
        await MultiAcquire([sheep])  # block 1 sheep
        await asyncio.sleep(2)  # 2 seconds
        print('Feed sheep')
        sheep.release()
    
    
    async def main():
        await asyncio.gather(
            spend('spend 1'),
            feed('feed 1'),
            spend('spend 2')
        )  # spend 2 times, feed 1 time
    
    
    if __name__ == "__main__":
        loop = asyncio.get_event_loop()
        loop.run_until_complete(main())
    

    输出:

    acquire gate (spend 2)
    acquire sheep (spend 2)
    acquire sheep (feed 1)
    Spend sheep through a gate
    release sheep (spend 2)
    release gate (spend 2)
    acquire sheep (spend 1)
    acquire gate (spend 1)
    Feed sheep
    release sheep (feed 1)
    Spend sheep through a gate
    release sheep (spend 1)
    release gate (spend 1)
    [Finished in 2.2s]