触发ExternalTask​​Sensor时气流阻塞

Question

谁能解释为什么Airflow的ExternalTask​​Sensor在调度时正常工作，而在Frontend触发时却不正常？我尝试了多个版本，总是得到相同的结果。任务在up_fro_reschedule上停止。

example_external_task_marker_child和example_external_task_marker_parent

Task.log

[2020-03-08 22:25:25,071] {taskinstance.py:866} INFO - Executing <Task(ExternalTaskSensor): child_task1> on 2020-03-08T21:16:27.647772+00:00
[2020-03-08 22:25:25,075] {standard_task_runner.py:53} INFO - Started process 58496 to run task
[2020-03-08 22:25:25,261] {logging_mixin.py:91} INFO - Running <TaskInstance: example_external_task_marker_child.child_task1 2020-03-08T21:16:27.647772+00:00 [None]> on host 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa
[2020-03-08 22:25:25,363] {taskinstance.py:934} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=example_external_task_marker_child
AIRFLOW_CTX_TASK_ID=child_task1
AIRFLOW_CTX_EXECUTION_DATE=2020-03-08T21:16:27.647772+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2020-03-08T21:16:27.647772+00:00
[2020-03-08 22:25:25,401] {external_task_sensor.py:119} INFO - Poking for example_external_task_marker_parent.parent_task on 2020-03-08T21:16:27.647772+00:00 ... 
[2020-03-08 22:25:25,538] {taskinstance.py:1110} INFO - Rescheduling task, marking task as UP_FOR_RESCHEDULE
[2020-03-08 22:25:29,903] {logging_mixin.py:91} INFO - [2020-03-08 22:25:29,902] {local_task_job.py:103} INFO - Task exited with return code 0
[2020-03-08 22:26:43,224] {taskinstance.py:656} INFO - Dependencies all met for <TaskInstance: example_external_task_marker_child.child_task1 2020-03-08T21:16:27.647772+00:00 [queued]>
[2020-03-08 22:26:43,275] {taskinstance.py:656} INFO - Dependencies all met for <TaskInstance: example_external_task_marker_child.child_task1 2020-03-08T21:16:27.647772+00:00 [queued]>
[2020-03-08 22:26:43,275] {taskinstance.py:845} INFO - 

Answer 1

默认情况下，ExternalTaskSensor将以与传感器DAG相同的执行日期来监视external_dag_id。当两个dag都以相同的时间表运行时，或者当您确切知道两者之间的时间间隔时，这将非常有用。

但是，当手动或由另一个dag触发一个dag时，您无法确定这两个dag中任何一个的确切执行日期。这就是为什么从前端触发失败的原因。

如果您用TriggerDagRunOperator触发了dag，则可以解决此问题。看看this question了解更多信息。