Question

我正在创建一个用于读取不断增长的日志文件的子流程。我将一个计数器（在列表内）传递给log_file_reader函数，如果该行有效，则将1附加到计数器列表。我每隔5秒检查主过程中的计数器。计数器在子过程中按预期增加，但在主过程中始终为0。我检查了柜台的ID;它在子流程和主流程中都是相同的。为什么主要流程中的计数器没有增加？如果我将计数器更改为counter = multiprocessing.Queue()并检查qsize()中的log_file_reader(...)或主线程，则一切正常。

import subprocess
import select
import multiprocessing
import time


def log_file_reader(filename, counter):
    f = subprocess.Popen(['tail', '-F',filename], stdout=subprocess.PIPE,stderr=subprocess.PIPE)
    p = select.poll()
    p.register(f.stdout)
    while True:
        if p.poll(1):
            line = f.stdout.readline().strip()
            if line:
                '''appends 1 to counter if line is valid'''
                counter.append(1)


def main():
    counter = list()  # initializes a counter in type list
    # starts up a process keep tailing file
    reader_process = multiprocessing.Process(target=log_file_reader, args=("/home/haifzhan/logfile.log", counter))
    reader_process.start()

    # main thread check the counter every 5 seconds
    while True:
        time.sleep(5)
        print "periodically check---counter:{0},id:{1}".format(len(counter), id(counter))


if __name__ == "__main__":
    # everything starts here
    main()

Answer 1

普通list对象不在进程之间共享，因此子进程中的counter实际上是与父进程中的counter完全不同的对象。您对其中的更改不会影响另一个。如果要在进程之间共享列表，则需要使用multiprocessing.Manager().list：

import subprocess
import select
import multiprocessing
import time


def log_file_reader(filename, counter):
    f = subprocess.Popen(['tail', '-F',filename], stdout=subprocess.PIPE,stderr=subprocess.PIPE)
    p = select.poll()
    p.register(f.stdout)
    while True:
        if p.poll(1):
            line = f.stdout.readline().strip()
            if line:
                '''appends 1 to counter if line is valid'''
                counter.append(1)


def main():
    m = multiprocessing.Manager()
    counter = m.list()  # initializes a counter in type list
    # starts up a process keep tailing file
    reader_process = multiprocessing.Process(target=log_file_reader, args=("/home/haifzhan/logfile.log", counter))
    reader_process.start()

    # main thread check the counter every 5 seconds
    while True:
        time.sleep(5)
        print "periodically check---counter:{0},id:{1}".format(len(counter), id(counter))


if __name__ == "__main__":
    # everything starts here
    main()

如果您只是将列表用作计数器，那么您也可以使用multiprocessing.Value而不是列表，它实际上是用于计数目的，并且不会＃＆＃ 39; t需要启动Manager进程：

import subprocess
import select
import multiprocessing
import time


def log_file_reader(filename, counter):
    f = subprocess.Popen(['tail', '-F',filename], stdout=subprocess.PIPE,stderr=subprocess.PIPE)
    p = select.poll()
    p.register(f.stdout)
    while True:
        if p.poll(1):
            line = f.stdout.readline().strip()
            if line:
                '''appends 1 to counter if line is valid'''
                with counter.get_lock():
                    counter.value += 1


def main():
    m = multiprocessing.Manager()
    counter = multiprocessing.Value('i', 0) # A process-safe int, initialized to 0
    # starts up a process keep tailing file
    reader_process = multiprocessing.Process(target=log_file_reader, args=("/home/haifzhan/logfile.log", counter))
    reader_process.start()

    # main thread check the counter every 5 seconds
    while True:
        time.sleep(5)
        with counter.get_lock():
            print "periodically check---counter:{0},id:{1}".format(counter.value, id(counter))

为什么子进程中的列表更改未显示在父进程中？

1 个答案: