我想为volumes
的{{1}}属性生成条目。我像这样创建DockerOperator:
DockerOperator
DAG由传递一些参数的REST调用触发。我想通过迭代这些参数来动态填充def create_docker_operator(dag):
# TODO: fill up volumes dynamically with data from REST paramters
volumes = [ "{{ task_instance.xcom_pull(key='run_path') }}:/workdir" ]
return DockerOperator(task_id="start_docker",
container_name="{{ task_instance.xcom_pull(key='run_folder') }}_{{ task_instance.xcom_pull(key='project_name') }}",
image="{{ task_instance.xcom_pull(key='run_env_docker_name') }}",
api_version="auto",
auto_remove=False,
xcom_push=True,
xcom_all=True,
volumes=volumes,
command="python /workdir/bin/python_caller.py /workdir/bin/{{ task_instance.xcom_pull(key='script_name') }}",
network_mode="bridge",
dag=dag)
with airflow.DAG("learning", default_args=default_args, schedule_interval=None) as dag:
create_run_task = PythonOperator(task_id="create_run", provide_context=True, python_callable=create_run, dag=dag)
prepare_data_task = PythonOperator(task_id="prepare_data", provide_context=True, python_callable=prepare_data, dag=dag)
start_learning_dockered_task = create_docker_operator(dag)
check_result_task = PythonOperator(task_id="check_result", provide_context=True, python_callable=check_result, dag=dag)
create_run_task >> prepare_data_task >> start_learning_dockered_task >> check_result_task
并将其附加到volumes
上。不幸的是,我没有此解决方案的上下文来访问REST调用中的参数。是否可以从传递的volumes
实例访问dag
来访问它?