在google composer中删除DAG - Airflow UI

Question

我想从Airflow用户界面中删除DAG，这在GCS/dags文件夹中不再可用。 我知道Airflow有一种“新的”方法可以从数据库中删除dags 在https://stackoverflow.com/a/49683543/5318634

中看到的airflow delete_dag my_dag_id命令

似乎在composer airflow版本中，delete_dag命令尚不支持。

请勿尝试：我也尝试使用airflow resetdb并且气流用户界面已经死亡

有没有办法删除当前不在gs://BUCKET/dags/文件夹中的dag？

Answer 1

我创建了一个DAG来清理UI。它从Airflow Variables中读取afDagID

from airflow import DAG
from airflow import models
from airflow.operators.mysql_operator import MySqlOperator
import logging
from datetime import datetime
from airflow.operators.dummy_operator import DummyOperator

dag = DAG('ManageAirFlow', description='Deletes Airflow DAGs from backend: Uses vars-  afDagID',
      schedule_interval=None,
      start_date=datetime(2018, 3, 20), catchup=False)

DeleteXComOperator = MySqlOperator(
  task_id='delete-xcom-record-task',
  mysql_conn_id='airflow_db',
  sql="DELETE from xcom where dag_id='{}'".format(models.Variable.get('afDagID')),
  dag=dag)

DeleteTaskOperator = MySqlOperator(
  task_id='delete-task-record-task',
  mysql_conn_id='airflow_db',
  sql="DELETE from task_instance where dag_id='{}'".format(models.Variable.get('afDagID')),
  dag=dag)

DeleteSLAMissOperator = MySqlOperator(
  task_id='delete-sla-record-task',
  mysql_conn_id='airflow_db',
  sql="DELETE from sla_miss where dag_id='{}'".format(models.Variable.get('afDagID')),
  dag=dag)

DeleteLogOperator = MySqlOperator(
  task_id='delete-log-record-task',
  mysql_conn_id='airflow_db',
  sql="DELETE from log where dag_id='{}'".format(models.Variable.get('afDagID')),
  dag=dag)

DeleteJobOperator = MySqlOperator(
  task_id='delete-job-record-task',
  mysql_conn_id='airflow_db',
  sql="DELETE from job where dag_id='{}'".format(models.Variable.get('afDagID')),
  dag=dag)

DeleteDagRunOperator = MySqlOperator(
  task_id='delete-dag_run-record-task',
  mysql_conn_id='airflow_db',
  sql="DELETE from dag_run where dag_id='{}'".format(models.Variable.get('afDagID')),
  dag=dag)

DeleteDagOperator = MySqlOperator(
  task_id='delete-dag-record-task',
  mysql_conn_id='airflow_db',
  sql="DELETE from dag where dag_id='{}'".format(models.Variable.get('afDagID')),
  dag=dag)



DeleteXComOperator >> DeleteTaskOperator >> DeleteSLAMissOperator >> DeleteLogOperator >> DeleteJobOperator >> DeleteDagRunOperator >> DeleteDagOperator

Answer 2

由于云作曲家使用最新的稳定版本，即1.9.0，因此删除dag的功能不可用。

然而，

文档中很少有删除dag的说明，如下所示：

 gcloud beta composer environments storage dags delete \
     --environment ENVIRONMENT_NAME \
     --location LOCATION \
     DAG_NAME.py 

但不幸的是，这不会从Airflow网络界面中删除DAG。

更多信息：https://cloud.google.com/composer/docs/how-to/using/managing-dags#deleting_a_dag

Answer 3

这将分两个步骤进行： 步骤1：-

首先，您必须使用命令从存储分区中删除 airflow_monitoring.py 文件。

gcloud composer环境存储dags删除--environment viu-etl-prod-composer --location us-central1 airflow_monitoring.py

步骤2：-
单击红叉检查，如图所示。