Question

我有一个 DAG，它的开始日期是 2021-04-02，计划间隔是每个月的第 3 天晚上 11 点。执行日期似乎工作正常，但 DAG 似乎每天在 2:30 持续运行。我似乎无法找出问题所在。

default_args = {
    'owner': DAG_OWNER_NAME,
    'email': ALERT_EMAIL_ADDRESSES,
    'email_on_failure': True,
    'email_on_retry': False,
    'start_date': datetime(2021, 4, 2),
    'retries': 0,
    #'sla': timedelta(seconds=30),
    'retry_delay': timedelta(minutes=1)
    }

dag = DAG(DAG_ID, default_args=default_args, schedule_interval='0 23 3 * *',max_active_runs=1,catchup=False)

执行似乎是在同一个任务实例中运行的，如下面缩短的混淆日志所示。

*** Reading local file: /home/lcg_airflow/airflow/logs/DAG_ID/TASK_ID/2021-04-03T23:00:00+00:00/1.log
[2021-05-03 23:00:20,576] {models.py:1359} INFO - Dependencies all met for <TaskInstance: DAG_ID.TASK_ID 2021-04-03T23:00:00+00:00 [queued]>
[2021-05-03 23:00:20,580] {models.py:1571} INFO - 
--------------------------------------------------------------------------------
Starting attempt 1 of 1
--------------------------------------------------------------------------------

[2021-05-03 23:00:20,590] {models.py:1593} INFO - Executing <Task(EmailOperator): TASK_ID> on 2021-04-03T23:00:00+00:00
[2021-05-03 23:00:20,590] {base_task_runner.py:118} INFO - Running: ['bash', '-c', 'airflow run DAG_ID TASK_ID 2021-04-03T23:00:00+00:00 --job_id 1945275 --raw -sd DAGS_FOLDER/DAG_ID.py --cfg_path /tmp/tmpuebs1mob']
[2021-05-03 23:00:21,071] {base_task_runner.py:101} INFO - Job 1945275: Subtask TASK_ID [2021-05-03 23:00:21,070] {settings.py:174} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=3600, pid=115477
[2021-05-03 23:00:21,625] {base_task_runner.py:101} INFO - Job 1945275: Subtask TASK_ID [2021-05-03 23:00:21,624] {__init__.py:51} INFO - Using executor CeleryExecutor
logs for task continue... 
[2021-05-03 23:00:25,577] {logging_mixin.py:95} INFO - [2021-05-03 23:00:25,576] {jobs.py:2527} INFO - Task exited with return code 0
[2021-05-04 02:30:51,314] {models.py:1359} INFO - Dependencies all met for <TaskInstance: DAG_ID.TASK_ID 2021-04-03T23:00:00+00:00 [queued]>
[2021-05-04 02:30:51,317] {models.py:1359} INFO - Dependencies all met for <TaskInstance: DAG_ID.TASK_ID 2021-04-03T23:00:00+00:00 [queued]>
[2021-05-04 02:30:51,317] {models.py:1571} INFO - 
--------------------------------------------------------------------------------
Starting attempt 1 of 1
--------------------------------------------------------------------------------

[2021-05-04 02:30:51,326] {models.py:1593} INFO - Executing <Task(EmailOperator): TASK_ID> on 2021-04-03T23:00:00+00:00
[2021-05-04 02:30:51,327] {base_task_runner.py:118} INFO - Running: ['bash', '-c', 'airflow run DAG_ID TASK_ID 2021-04-03T23:00:00+00:00 --job_id 1946386 --raw -sd DAGS_FOLDER/DAG_ID.py --cfg_path /tmp/tmpyccmvf2b']
[2021-05-04 02:30:51,790] {base_task_runner.py:101} INFO - Job 1946386: Subtask TASK_ID [2021-05-04 02:30:51,790] {settings.py:174} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=3600, pid=42760
[2021-05-04 02:30:52,363] {base_task_runner.py:101} INFO - Job 1946386: Subtask TASK_ID [2021-05-04 02:30:52,362] {__init__.py:51} INFO - Using executor CeleryExecutor
logs for task continue and repeat every day at 2:30 AM...

有人知道这是什么原因造成的吗？

每月 DAG 计划在每天凌晨 2:30 运行

0 个答案: