多处理超时?

时间:2018-02-19 16:57:14

标签: python windows python-2.7 timeout multiprocessing

我已经搜索过StackOverflow,虽然我发现了很多问题,但我还没有找到适合我情况的答案/不是一个强大的python程序员来调整他们的答案以适应我的需要。

我在这看来无济于事:

kill a function after a certain time in windows

Python: kill or terminate subprocess when timeout

signal.alarm replacement in Windows [Python]

我正在使用多处理来一次运行多个SAP窗口来提取报告。设置为每5分钟按计划运行一次。每隔一段时间,其中一个报告因GUI界面而停滞不前,永远不会结束。我没有得到错误或异常,它只会永远失速。我想要的是有一个超时功能,在SAP执行的这部分代码中,如果它需要超过4分钟,它会超时,关闭SAP,跳过剩下的代码,等待下一个计划报告时间。

我正在使用Windows Python 2.7

import multiprocessing
from multiprocessing import Manager, Process
import time
import datetime

### OPEN SAP ###
def start_SAP():
   print 'opening SAP program'

### REPORTS IN SAP ###
def report_1(q, lock):
   while True:  # logic to get shared queue
       if not q.empty():
           lock.acquire()
           k = q.get()
           time.sleep(1)
           lock.release()
           break
       else:
           time.sleep(1)
   print 'running report 1'

def report_2(q, lock):
   while True:  # logic to get shared queue
       if not q.empty():
           lock.acquire()
           k = q.get()
           time.sleep(1)
           lock.release()
           break
       else:
           time.sleep(1)
   print 'running report 2'

def report_3(q, lock):
   while True:  # logic to get shared queue
       if not q.empty():
           lock.acquire()
           k = q.get()
           time.sleep(1)
           lock.release()
           break
       else:
           time.sleep(1)

   time.sleep(60000) #mimicking the stall for report 3 that takes longer than allotted time
   print 'running report 3'

def report_N(q, lock):
   while True:  # logic to get shared queue
       if not q.empty():
           lock.acquire()
           k = q.get()
           time.sleep(1)
           lock.release()
           break
       else:
           time.sleep(1)
   print 'running report N'

### CLOSES SAP ###
def close_SAP():
   print 'closes SAP'

def format_file():
   print 'formatting files'

def multi_daily_pull():

    lock = multiprocessing.Lock()  # creating a lock in multiprocessing

    shared_list = range(6)  # creating a shared list for all functions to use
    q = multiprocessing.Queue()  # creating an empty queue in mulitprocessing
    for n in shared_list:  # putting list into the queue
        q.put(n)
    print 'Starting process at ', time.strftime('%m/%d/%Y %H:%M:%S')

    print 'Starting SAP Pulls at ', time.strftime('%m/%d/%Y %H:%M:%S')

    StartSAP = Process(target=start_SAP)
    StartSAP.start()
    StartSAP.join()

    report1= Process(target=report_1, args=(q, lock))
    report2= Process(target=report_2, args=(q, lock))
    report3= Process(target=report_3, args=(q, lock))
    reportN= Process(target=report_N, args=(q, lock))

    report1.start()
    report2.start()
    report3.start()
    reportN.start()

    report1.join()
    report2.join()
    report3.join()
    reportN.join()

    EndSAP = Process(target=close_SAP)
    EndSAP.start()
    EndSAP.join()

    formatfile = Process(target=format_file)
    formatfile .start()
    formatfile .join()

if __name__ == '__main__':
    multi_daily_pull()

1 个答案:

答案 0 :(得分:2)

执行所需操作的一种方法是使用Process.join()方法接受的可选超时参数。这将使它仅在最长时间内阻止调用线程。

我还设置了每个Process实例的daemon属性,这样即使其中一个进程仍在“运行”(或已挂起),您的主线程也能够终止。

最后一点,您不需要multiprocessing.Lock来控制对multiprocessing.Queue的访问,因为它们会自动处理这方面的事情,所以我将其删除了。您可能仍然希望有一个由于某些其他原因,例如控制对stdout的访问,因此从各种进程打印到它不会重叠并弄乱输出到屏幕的内容。

import multiprocessing
from multiprocessing import Process
import time
import datetime

def start_SAP():
    print 'opening SAP program'

### REPORTS IN SAP ###
def report_1(q):
    while True:  # logic to get shared queue
        if q.empty():
            time.sleep(1)
        else:
            k = q.get()
            time.sleep(1)
            break

    print 'report 1 finished'

def report_2(q):
    while True:  # logic to get shared queue
        if q.empty():
            time.sleep(1)
        else:
            k = q.get()
            time.sleep(1)
            break

    print 'report 2 finished'

def report_3(q):
    while True:  # logic to get shared queue
        if q.empty():
            time.sleep(1)
        else:
            k = q.get()
            time.sleep(60000) # Take longer than allotted time
            break

    print 'report 3 finished'


def report_N(q):
    while True:  # logic to get shared queue
        if q.empty():
            time.sleep(1)
        else:
            k = q.get()
            time.sleep(1)
            break

    print 'report N finished'

def close_SAP():
    print 'closing SAP'

def format_file():
    print 'formatting files'

def multi_daily_pull():
    shared_list = range(6)  # creating a shared list for all functions to use
    q = multiprocessing.Queue()  # creating an empty queue in mulitprocessing
    for n in shared_list:  # putting list into the queue
        q.put(n)
    print 'Starting process at ', time.strftime('%m/%d/%Y %H:%M:%S')

    print 'Starting SAP Pulls at ', time.strftime('%m/%d/%Y %H:%M:%S')

    StartSAP = Process(target=start_SAP)
    StartSAP.start()
    StartSAP.join()

    report1 = Process(target=report_1, args=(q,))
    report1.daemon = True
    report2 = Process(target=report_2, args=(q,))
    report2.daemon = True
    report3 = Process(target=report_3, args=(q,))
    report3.daemon = True
    reportN = Process(target=report_N, args=(q,))
    reportN.daemon = True

    report1.start()
    report2.start()
    report3.start()
    reportN.start()

    report1.join(30)
    report2.join(30)
    report3.join(30)
    reportN.join(30)

    EndSAP = Process(target=close_SAP)
    EndSAP.start()
    EndSAP.join()

    formatfile = Process(target=format_file)
    formatfile .start()
    formatfile .join()

if __name__ == '__main__':
    multi_daily_pull()