按特定顺序执行任务

时间:2014-06-11 00:47:07

标签: python multithreading python-2.7 task-queue

我有这个循环:

listGames = []
for home in range(totalPlayers - 1):
    for away in range(home + 1, totalPlayers):
        listGames.append((home, away))

print listGames
match_num = 1
for game in listGames:
    player1 = listPlayers[game[0]]
    player2 = listPlayers[game[1]]
    do_stuff(player1, player2)

当有很多玩家时,这个循环可能需要相当长的时间,因此希望使用线程来更快地完成循环。但是,player1和player2是类的实例,因此同时使用它们的东西会很糟糕。编辑:执行这些“任务”的顺序无关紧要。

我发现http://www.troyfawkes.com/learn-python-multithreading-queues-basics/似乎正是我想要的,但我不确定如何调整它以确保一次只运行一个类/玩家的实例

(简单)示例:

totalPlayers = 4

0, 1, 2, 3
listGames = [(0, 1), (0, 2), (0, 3), (1, 2), (1, 3), (2, 3)]

所以,游戏(0,1),(2,3)可以同时执行,但其他人必须等到这些完成后

提示/想法?

2 个答案:

答案 0 :(得分:2)

除非do_stuff()与IO绑定,否则这可能会使您的代码更慢because of the global interpreter lock。根据你的声明,当有很多玩家时,这可能需要时间,"我倾向于认为你的程序可能是CPU限制的 - 在这种情况下,多线程可能会伤害你的性能。

说到你原来的问题,你要从两个元素子集中要求exact cover你的一组玩家 - 不幸的是,这是NP完全的。

答案 1 :(得分:1)

这是一个示例程序,显示了一种可以执行此操作的方法。我们的想法是创建一个multiprocessing.Pool来同时运行do_stuff的许多实例。我们还维护set来跟踪do_stuff实例当前正在处理的所有玩家,这样我们就不会同时处理同一个玩家。当do_stuff完成其工作时,它告诉父进程它已经完成了玩家,因此可以处理使用这些玩家的新任务。

import time
import multiprocessing
from Queue import Empty

listGames = [(0, 1), (0, 2), (0, 3), (1, 2), (1, 3), (2, 3)] 

def do_stuff(players):
    player1, player2 = players
    print("player1 {} player2 {}".format(player1, player2))
    time.sleep(5)
    # Imagine some other stuff happens here.
    print ("now done with {} and {}".format(player1, player2))
    q.put((player1, player2))

if __name__ == "__main__":
    q = multiprocessing.Queue()
    pool = multiprocessing.Pool()
    gamesSet = set(listGames)  # Convert to a set for efficiency reasons.
    running = set()  # This keeps track of players being processed.
    while gamesSet:
        to_remove = []
        for player in gamesSet:  
            if player[0] not in running and player[1] not in running:
                running.add(player[0])
                running.add(player[1])
                pool.apply_async(do_stuff, (player,))
                to_remove.append(player)
        for player in to_remove:
            gamesSet.remove(player)
        while True:
           # Find out if we're done processing any players.
           try:
               done = q.get_nowait()
               running.remove(done[0])
               running.remove(done[1])
           except Empty:
               break
    pool.close()
    pool.join()

输出:

dan@dantop2:~$ ./mult.py 
player1 0 player2 1
player1 2 player2 3
now done with 0 and 1
now done with 2 and 3
player1 1 player2 2
player1 0 player2 3
now done with 0 and 3
now done with 1 and 2
player1 1 player2 3
player1 0 player2 2
now done with 1 and 3
now done with 0 and 2