Question

我创建了一个字典，我将id与子进程相关联。类似的东西：

cmd = "ls"
processes[id] = subprocess.Popen([cmd], shell=True, stdout=subprocess.PIPE)

然后我用这个流程图作为输入调用一个方法，检查哪个流程已经完成。如果该过程结束，我会检查进程的stdout.read()以查找特定的字符串匹配。

问题有时stdout.read()返回一个空值，这会导致字符串匹配问题。

示例代码：

#Create a map
processes[id] = subprocess.Popen([cmd], shell=True, stdout=subprocess.PIPE)
...
#Pass that map to a method which checks which processes have finished
completedProcesses(processes)

def completedProcesses(processes):
    processList = []
    for id,process in processes.iteritems():
        if process.poll() is not None:
            #If some error in process stdout then print id
            verifySuccessStatus(id, processes[id])
            processList.add(id)

def verifySuccessStatus(id, process):
    file=open(FAILED_IDS_FILE, 'a+')
    buffer =  process.stdout.read() #This returns empty value sometime
    if 'Error' not in buffer:
        file.write(id)
        file.write('\n')
    file.close()

我是python的新手，我可能缺少对子进程的一些内部功能理解

Answer 1

至少有两个问题：

没有必要多次拨打process.stdout.read()。直到EOF，.read()才会返回。它返回一个空字符串，表示之后的EOF。
您应该在进程仍在运行时从管道读取否则如果它们生成足够的输出来填充OS管道缓冲区（在我的Linux机器上大约65K），它们可能会挂起。

如果要同时运行多个外部进程并在完成后检查其输出，请参阅this answer that shows "thread pool" and async.io solutions。

Answer 2

根据ls的示例命令判断，您的问题可能是由stdout管道填满造成的。使用process.communicate()方法可以为您处理这种情况，因为您不需要多次写入stdin。

# Recommend the future print function for easier file writing.
from __future__ import print_function

# Create a map
# Keeping access to 'stderr' is generally recommended, but not required.
# Also, if you don't know you need 'shell=True', it's safer practice not to use it.
processes[id] = subprocess.Popen(
    [cmd],
    shell=True,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    )
...
#Pass that map to a method which checks which processes have finished
check_processes(processes)

def check_processes(processes):
    process_ids = []

    # 'id' is a built-in function in python, so it's safer to use a different name.
    for idx, process in processes.iteritems():
        # When using pipes, communicate() will handle the case of the pipe
        # filling up for you.
        stdout, stderr = process.communicate()
        if not is_success(stdout):
            write_failed_id(idx)
        process_ids.append(idx)

def is_success(stdout):
    return 'Error' not in stdout

def write_failed_id(idx):
    # Recommend using a context manager when interacting with files.
    # Also, 'file' is a built-in function in python.
    with open(FAILED_IDS_FILE, 'a+') as fail_file:
        # The future print function makes file printing simpler.
        print(idx, file=fail_file)

Answer 3

您只需阅读stdout并查找＆＃34;错误＆＃34;。也许您也应该查看stderr：

processes[id] = subprocess.Popen(
    [cmd], 
    shell=True, 
    stdout=subprocess.PIPE, 
    stderr=subprocess.STDOUT,
    )

来自subprocess docs：

子进程。的 STDOUT

可以用作Popen的stderr参数的特殊值，表示标准错误应该与标准输出放在同一个句柄中。

进程可能意外失败，返回没有stdout但返回非零返回码。您可以使用process.returncode。
进行检查

POPEN。的返回码

子返回码，由poll（）和wait（）设置（间接由communic（）设置）。 “无”值表示该进程尚未终止。

负值-N表示孩子被信号N终止（仅限Unix）。

stdout.read（）从完成的子进程有时返回空？

3 个答案: