合并和同步stdout和stderr?

时间:2011-02-13 14:02:52

标签: python subprocess stdout stderr

说我正在使用以下命令从python脚本运行exe:

subprocess.call(cmdArgs,stdout=outf, stderr=errf)

当outf和errf是文本文件的文件描述符时。

有什么方法可以在它上面生成stdout和stderr的合并并同步的文本文件? 它应该用时间和来源格式化(我们/错误)。

感谢

2 个答案:

答案 0 :(得分:4)

这有点棘手,因为你需要在子进程运行时轮询stdout和stderr文件描述符,以获得准确的时间戳。您还需要将输出切割为行列表,以便最终结果可以轻松合并和排序。您可以在阅读时轻松合并两个流,但这不是问题的一部分。

我写得很快,但它可以变得更干净,更紧凑:

import datetime
import os
import select
import subprocess

class Stream(object):

    def __init__(self, name, impl):
        self._name = name
        self._impl = impl
        self._buf = ''
        self._rows = []

    def fileno(self):
        "Pass-through for file descriptor."
        return self._impl.fileno()

    def read(self, drain=0):
        "Read from the file descriptor. If 'drain' set, read until EOF."
        while self._read() is not None:
            if not drain:
                break

    def _read(self):
        "Read from the file descriptor"
        fd = self.fileno()
        buf = os.read(fd, 4096)
        if not buf:
            return None
        if '\n' not in buf:
            self._buf += buf
            return []

        # prepend any data previously read, then split into lines and format
        buf = self._buf + buf
        tmp, rest = buf.rsplit('\n', 1)
        self._buf = rest
        now = datetime.datetime.now().isoformat()
        rows = tmp.split('\n')
        self._rows += [(now, '%s %s: %s' % (self._name, now, r)) for r in rows]

def run(cmd, timeout=0.1):
    """
    Run a command, read stdout and stderr, prefix with timestamp, and
    return a dict containing stdout, stderr and merged.
    """
    PIPE = subprocess.PIPE
    proc = subprocess.Popen(cmd, stdout=PIPE, stderr=PIPE)
    streams = [
        Stream('stdout', proc.stdout),
        Stream('stderr', proc.stderr)
        ]
    def _process(drain=0):
        res = select.select(streams, [], [], timeout)
        for stream in res[0]:
            stream.read(drain)

    while proc.returncode is None:
        proc.poll()
        _process()
    _process(drain=1)

    # collect results, merge and return
    result = {}
    temp = []
    for stream in streams:
        rows = stream._rows
        temp += rows
        result[stream._name] = [r[1] for r in rows]
    temp.sort()
    result['merged'] = [r[1] for r in temp]
    return result

res = run(['ls', '-l', '.', 'xyzabc'])
for key in ('stdout', 'stderr', 'merged'):
    print 
    print '\n'.join(res[key])
    print '-'*40

示例输出:

stdout 2011-03-03T19:30:44.838145: .:
stdout 2011-03-03T19:30:44.838145: total 0
stdout 2011-03-03T19:30:44.838338: -rw-r--r-- 1 pat pat 0 2011-03-03 19:30 bar
stdout 2011-03-03T19:30:44.838518: -rw-r--r-- 1 pat pat 0 2011-03-03 19:30 foo
----------------------------------------

stderr 2011-03-03T19:30:44.837189: ls: cannot access xyzabc: No such file or directory
----------------------------------------

stderr 2011-03-03T19:30:44.837189: ls: cannot access xyzabc: No such file or directory
stdout 2011-03-03T19:30:44.838145: .:
stdout 2011-03-03T19:30:44.838145: total 0
stdout 2011-03-03T19:30:44.838338: -rw-r--r-- 1 pat pat 0 2011-03-03 19:30 bar
stdout 2011-03-03T19:30:44.838518: -rw-r--r-- 1 pat pat 0 2011-03-03 19:30 foo
----------------------------------------

答案 1 :(得分:0)

您可以将subprocess.STDOUT作为stderr的{​​{1}}参数合并,但我不知道它们是否会按时间和来源进行格式化。