Question

从python代码调用/触发Luigi Task时遇到了麻烦。

基本上我需要触发一个luigi任务，就像我们在命令行上一样，但是从python代码
我使用supbrocess.popen使用shell调用luigi任务命令
我有一个名为test.py的测试代码，并在模块中有一个测试类 task_scheduler.py包含我的luigi任务（两个模块位于同一位置/目录）

＆＃13;

        import luigi
        class TestClass(luigi.Task):
            # param = luigi.DictParameter(default=dict())

            def requires(self):
                print "I am TestClass req"

            def run(self):
                with open('myfile.txt', 'w') as f:
                    f.write("asasasas")

                print "I am TestClass run"

＆＃13;

import subprocess

p = subprocess.Popen("python -m luigi --module task_scheduler TestClass", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

print p.pid
(output, err) = p.communicate()

print "-------------O/P-------------"
print output
print "-------------error-------------"
print err

＆＃13;

但我收到的错误是

＆＃13;

52688
-------------O/P-------------

-------------error-------------
ERROR: Uncaught exception in luigi
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/luigi/retcodes.py", line 61, in run_with_retcodes
    worker = luigi.interface._run(argv)['worker']
  File "/Library/Python/2.7/site-packages/luigi/interface.py", line 238, in _run
    return _schedule_and_run([cp.get_task_obj()], worker_scheduler_factory)
  File "/Library/Python/2.7/site-packages/luigi/interface.py", line 172, in _schedule_and_run
    not(lock.acquire_for(env_params.lock_pid_dir, env_params.lock_size, kill_signal))):
  File "/Library/Python/2.7/site-packages/luigi/lock.py", line 82, in acquire_for
    my_pid, my_cmd, pid_file = get_info(pid_dir)
  File "/Library/Python/2.7/site-packages/luigi/lock.py", line 67, in get_info
    pid_file = os.path.join(pid_dir, hashlib.md5(cmd_hash).hexdigest()) + '.pid'
TypeError: must be string or buffer, not None

＆＃13;

任何人都可以建议我在这里做错了吗？命令＆＃34; python -m luigi --module task_scheduler TestClass＆＃34;如果我使用shell提示

，则效果很好

Answer 1

我刚尝试从命令行执行test.py. 似乎当我使用IDE运行（PyCharm）时，只有它给出了这个问题

Answer 2

这已在版本2.2.0中修复。检查github问题#1654

错误：luigi中未捕获的异常（TypeError：必须是字符串或缓冲区，而不是None）

2 个答案: