Question

我有一个芹菜任务，用subprocess调用Django应用程序外部的其他python脚本。这个程序里面有一些print，我想把这些打印放在我的芹菜日志文件或我的数据库中。当我在Django CELERY_ALWAYS_EAGER = True文件中设置settings.py时，一切正常。如果我没有设置此设置，celery任务日志子进程stdout仅在退出时。似乎p.stdout.readline()正在阻止。

run-test.py是一个漫长的过程，几分钟，但它打印出它正在做的事情。我想抓住这个。

@shared_task
def run_tests(scenario_path, vu):
    basedir = os.path.abspath(os.path.dirname(__file__))
    config_path = '%s/../../scripts/config.ini' % basedir
    cmd = ['python', '%s/../../scripts/aws/run-test.py' % basedir, '%s' % config_path, scenario_path, str(vu), str(2)]
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
    while True:
        line = p.stdout.readline()
        if line != '':
            logger.info(line)
        else:
            return

Answer 1

我发现这非常有用，使用select进行轮询而不是在readline上阻塞。

https://gist.github.com/bgreenlee/1402841

child = subprocess.Popen(popenargs, stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE, **kwargs)

log_level = {child.stdout: stdout_log_level,
             child.stderr: stderr_log_level}

def check_io():
    ready_to_read = select.select([child.stdout, child.stderr], [], [], 1000)[0]
    for io in ready_to_read:
        line = io.readline()
        logger.log(log_level[io], line[:-1])

# keep checking stdout/stderr until the child exits
while child.poll() is None:
    check_io()

check_io()  # check again to catch anything after the process exits

Celery任务子进程stdout来记录

1 个答案: