如果我不使用这些参数,为什么Python会改变函数的性质呢?

时间:2017-03-23 21:50:29

标签: python airflow

我手头有一种非常奇怪的python行为。

通过Airflow调用以下功能(AirBnB' s调度程序框架)PythonOperator TypeError: my_fun() takes exactly 1 argument (0 given)神秘失败:

def my_fun(a, **kwargs):
    print('line')

但是如果我使用这样的论点:

def my_fun(a, **kwargs):
    # At run time it appears the framework is 
    # giving this function 20 arguments.
    # Without these following two meaningless lines, 
    # python doesn't think my function can take those
    assert(a is not None)
    assert (kwargs is not None)
    print('line')

运行良好!如果我使用不带参数的函数,它将失败并显示takes exactly 0 argument (20 given)。这是一个更详细的堆栈跟踪:

ERROR - my_fun() takes exactly 1 argument (0 given)
Traceback (most recent call last):
  File "/root/venv/airflow/bin/airflow", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/root/venv/airflow/src/airflow/airflow/bin/airflow", line 17, in <module>
    args.func(args)
  File "/root/venv/airflow/src/airflow/airflow/bin/cli.py", line 287, in test
    ti.run(force=True, ignore_dependencies=True, test_mode=True)
  File "/root/venv/airflow/src/airflow/airflow/models.py", line 978, in run
    result = task_copy.execute(context=context)
  File "/root/venv/airflow/src/airflow/airflow/operators/python_operator.py", line 65, in execute
    return_value = self.python_callable(*self.op_args, **self.op_kwargs)

我完全不喜欢。更是如此,因为一个看似相同的结构运行良好:

def my_thing(a, **kwargs):
    thing()

它可能是什么?

EDIT 以下是Airflow中的任务设置:

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2016, 4, 25)
}

dag = DAG('some_dag',
          default_args=default_args,
          schedule_interval=timedelta(hours=1),
          max_active_runs=1,
          concurrency=1)

def runs_fine(execution_date, **kwargs):
    print('fine')

runs_fine_task = PythonOperator(task_id='run_fine',
                                    concurrency=1,
                                    max_active_runs=1,
                                    retries=0,
                                    provide_context=True,
                                    python_callable=runs_fine,
                                    dag=dag)

def fails(execution_date, **kwargs):
    print('fails')

fails_task = PythonOperator(task_id='I_fail',
                               concurrency=1,
                               max_active_runs=1,
                               retries=0,
                               provide_context=True,
                               python_callable=fails,
                               dag=dag)

0 个答案:

没有答案