多处理脚本,用于扫描新值并使队列无法正常工作

时间:2016-08-16 18:32:49

标签: python multiprocessing

这是我的剧本:

# globals
MAX_PROCESSES = 50
my_queue = Manager().Queue() # queue to store our values
stop_event = Event() # flag which signals processes to stop
my_pool = None

def my_function(var):
   while not stop_event.is_set():
        #this script will run forever for each variable found
        return


def var_scanner():
    # Since `t` could have unlimited size we'll put all `t` value in queue 
    while not stop_event.is_set(): # forever scan `values` for new items
        x = Variable.objects.order_by('var').values('var__var')
        for t in x:
            t = t.values()
            my_queue.put(t)
        time.sleep(10) 

try:
    var_scanner = Process(target=var_scanner)
    var_scanner.start()
    my_pool = Pool(MAX_PROCESSES)

    while not stop_event.is_set():
        try: #  if queue isn't empty, get value from queue and create new process
             var = my_queue.get_nowait() # getting value from queue 
             p = Process(target=my_function, args=("process-%s" % var))
             p.start()
        except Queue.Empty:
             print "No more items in queue"

except KeyboardInterrupt as stop_test_exception:
     print(" CTRL+C pressed. Stopping test....")
     stop_event.set()

但是我不认为这个剧本正是我想要的。这是我编写脚本时所寻找的内容。我想让它在" Variables"中扫描变量。表,添加"新"变量如果它们不存在于队列中,则运行" my_function"对于队列中的每个变量。

我相信我对很多人都有WAYYYY,而不是stop_event.is_set()函数。因为现在它只打印出#34;队列中没有更多的项目"大约一百万次。

请帮助!! :)

0 个答案:

没有答案