我手头有一种非常奇怪的python行为。
通过Airflow调用以下功能(AirBnB' s调度程序框架)PythonOperator
TypeError: my_fun() takes exactly 1 argument (0 given)
神秘失败:
def my_fun(a, **kwargs):
print('line')
但是如果我使用这样的论点:
def my_fun(a, **kwargs):
# At run time it appears the framework is
# giving this function 20 arguments.
# Without these following two meaningless lines,
# python doesn't think my function can take those
assert(a is not None)
assert (kwargs is not None)
print('line')
运行良好!如果我使用不带参数的函数,它将失败并显示takes exactly 0 argument (20 given)
。这是一个更详细的堆栈跟踪:
ERROR - my_fun() takes exactly 1 argument (0 given)
Traceback (most recent call last):
File "/root/venv/airflow/bin/airflow", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/root/venv/airflow/src/airflow/airflow/bin/airflow", line 17, in <module>
args.func(args)
File "/root/venv/airflow/src/airflow/airflow/bin/cli.py", line 287, in test
ti.run(force=True, ignore_dependencies=True, test_mode=True)
File "/root/venv/airflow/src/airflow/airflow/models.py", line 978, in run
result = task_copy.execute(context=context)
File "/root/venv/airflow/src/airflow/airflow/operators/python_operator.py", line 65, in execute
return_value = self.python_callable(*self.op_args, **self.op_kwargs)
我完全不喜欢。更是如此,因为一个看似相同的结构运行良好:
def my_thing(a, **kwargs):
thing()
它可能是什么?
EDIT 以下是Airflow中的任务设置:
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': datetime(2016, 4, 25)
}
dag = DAG('some_dag',
default_args=default_args,
schedule_interval=timedelta(hours=1),
max_active_runs=1,
concurrency=1)
def runs_fine(execution_date, **kwargs):
print('fine')
runs_fine_task = PythonOperator(task_id='run_fine',
concurrency=1,
max_active_runs=1,
retries=0,
provide_context=True,
python_callable=runs_fine,
dag=dag)
def fails(execution_date, **kwargs):
print('fails')
fails_task = PythonOperator(task_id='I_fail',
concurrency=1,
max_active_runs=1,
retries=0,
provide_context=True,
python_callable=fails,
dag=dag)