Question

是否有一种简单的方法可以跟踪进程以及由其启动的任何子进程的CPU时间？

我尝试对multiprocessing.Process进行子类化以计算任意函数，例如：

import time
from multiprocessing import Process

class TimedProcess(Process):

    daemon = True

    def __init__(self, *args, **kwargs):
        super(TimedProcess, self).__init__(*args, **kwargs)
        self.t0 = time.clock()

    @property
    def duration_seconds(self):
        return time.clock() - self.t0

p = TimedProcess(target=my_long_running_func)
p.start()
while p.is_alive():
    print p.duration_seconds
    time.sleep(1)

然而，当我试图计算涉及Scikits-learn或其他涉及c-extensions或子进程的代码的函数时，我发现我的duration_sections经常报告0，或者只是几秒钟，即使代码会跑几个小时。我该如何解决这个问题？

Answer 1

您的代码几乎会打印CPU时间，但您在父进程中调用time.clock()而不是子进程。通过使用multiprocessing.Pipe，您可以将值从子进程传递到父进程：

import time
from threading import Thread
from multiprocessing import Process, Pipe

class TimedProcess(Process):

    daemon = True

    def __init__(self, *args, **kwargs):
        super(TimedProcess, self).__init__(*args, **kwargs)
        self.parent_conn, self.child_conn = Pipe()
        self.child_finished = False
        self._duration = 0.0

    def get_duration(self):
        if not self.child_finished:
            self.parent_conn.send(None)
            result = self.parent_conn.recv()
            if result == 'done':
                self.child_finished = True
            else:
                self._duration = result
        return self._duration

    def run(self):
        try:
            t0 = time.clock()
            Thread(target=self._run).start()
            while True:
                request = self.child_conn.recv()
                self.child_conn.send(time.clock() - t0)
                if request == 'stop':
                    break
        finally:
            self.child_conn.send('done')

    def _run(self):
        try:
            super(TimedProcess, self).run()
        finally:
            self.parent_conn.send('stop')

p = TimedProcess(target=my_long_running_func)
p.start()
while p.is_alive():
    time.sleep(1)
    print p.get_duration()

跟踪Python进程和子进程的CPU时间

1 个答案: