从DAG访问气流上下文

时间:2020-10-28 07:24:25

标签: python airflow

我想为volumes的{​​{1}}属性生成条目。我像这样创建DockerOperator:

DockerOperator

DAG由传递一些参数的REST调用触发。我想通过迭代这些参数来动态填充def create_docker_operator(dag): # TODO: fill up volumes dynamically with data from REST paramters volumes = [ "{{ task_instance.xcom_pull(key='run_path') }}:/workdir" ] return DockerOperator(task_id="start_docker", container_name="{{ task_instance.xcom_pull(key='run_folder') }}_{{ task_instance.xcom_pull(key='project_name') }}", image="{{ task_instance.xcom_pull(key='run_env_docker_name') }}", api_version="auto", auto_remove=False, xcom_push=True, xcom_all=True, volumes=volumes, command="python /workdir/bin/python_caller.py /workdir/bin/{{ task_instance.xcom_pull(key='script_name') }}", network_mode="bridge", dag=dag) with airflow.DAG("learning", default_args=default_args, schedule_interval=None) as dag: create_run_task = PythonOperator(task_id="create_run", provide_context=True, python_callable=create_run, dag=dag) prepare_data_task = PythonOperator(task_id="prepare_data", provide_context=True, python_callable=prepare_data, dag=dag) start_learning_dockered_task = create_docker_operator(dag) check_result_task = PythonOperator(task_id="check_result", provide_context=True, python_callable=check_result, dag=dag) create_run_task >> prepare_data_task >> start_learning_dockered_task >> check_result_task 并将其附加到volumes上。不幸的是,我没有此解决方案的上下文来访问REST调用中的参数。是否可以从传递的volumes实例访问dag来访问它?

0 个答案:

没有答案