如何消除产生气流DAG的错误

时间:2020-09-26 06:41:46

标签: airflow

创建dag出错 root / .venv / lib / python3.6 / site-packages / airflow / models / dag.py:1342:PendingDeprecationWarning:无法将请求的任务添加到DAG,因为DAG中已经有一个具有task_id create_tag_template_field_result的任务。从Airflow 2.0开始,尝试覆盖任务将引发异常。

    default_args = {
        'owner':'airflow',
        'depend_on_past': False,
        'start_date':  datetime(2018, 11, 5, 10, 00, 00),
        'retries':1,
        'retry_delay': timedelta(minutes= 1)
    }

    def get_activated_sources():
        request = "SELECT * FROM users"
        pg_hook = PostgresHook(postgre_conn_id="postgres", schema="postgres")
        connection = pg_hook.get_conn()
        cursor = connection.cursor()
        cursor.execute(request)
        sources = cursor.fetchall

        for source in sources:
                print( "Source: {0}} activated {1}".format(source[0], source[1]))
        
        return sources

   with DAG('hook_dag', 
            default_args=default_args,
            schedule_interval= '@once',
            catchup=False
           ) as dag:
     
     start_task = DummyOperator(task_id='start_task')
     hook_task = PythonOperator(task_id='hook_task', 
                                python_callable=get_activated_sources)
     
     start_task >> hook_task

如何解决问题?请帮助我

0 个答案:

没有答案