Question

我已经通过Airflow Admin UI定义了SSH连接。但是，我仅在UI中定义服务帐户，主机和端口。我正在第一个任务实例中检索密码，并且需要在第二个任务实例中使用密码更新SSH连接，并在第三个任务实例中使用它。

t1：调用R函数以检索svc帐户的密码（存储在xcom_push中）
t2：使用此密码更新SSH连接（我正在使用 SSHHook）ssh02.password =密码（通过xcom_pull检索）
t3：使用以前更新的连接（ssh02）调用服务器

当前t1和t2可以正常工作，但是t3失败，因为密码没有更新，并且它正在寻找基于.ssh密钥文件的身份验证。有人可以建议如何实现吗？

这是我的代码段：

    from airflow import models
    from airflow.contrib.operators.ssh_operator import SSHOperator
    from airflow.operators.python_operator import PythonOperator
    from airflow.operators.bash_operator import BashOperator
    from datetime import datetime, timedelta
    from airflow.contrib.hooks.ssh_hook import SSHHook
    from airflow.models import Variable
    from airflow.models import Connection
    from airflow.settings import Session
    from airflow.utils import db
    from airflow.utils.db import provide_session
    from airflow import DAG
    import logging
    import os

    svcpassword = 'XXXX'

    logging.getLogger().setLevel(logging.DEBUG)


    ssh01 = SSHHook(ssh_conn_id='ssh_conn1')
    ssh02 = SSHHook(ssh_conn_id='ssh_conn2')


    default_args = {
     'owner': 'user',
     'depends_on_past': False,
     'start_date': datetime.now(),
     'email': ['abcd@gmail.com'],
     'email_on_failure': True,
     'email_on_retry': True,
     'retries': 1,
     'retry_delay':timedelta(minutes=1)
  }


   dag = DAG('dag_POC', default_args=default_args, 
   schedule_interval="@once")

   path1 = '/home/user/R_samplescript'


   t1 = SSHOperator(
        task_id='SSHTask',
        command='Rscript '+path1+'.R',
        ssh_hook=ssh01,
        params={},retries =1 ,
        do_xcom_push = True,
        dag = dag
      )

  def create_new_connection(**kwargs):
       ti = kwargs['ti']
       pwd = ti.xcom_pull(task_ids='SSHTask')
       password = str(pwd).replace("\\n","\n")
       password = password[password.find(' ')+1 : ]
       password = password.strip()
       svcpassword = password
       db.merge_conn( models.Connection(
                   conn_id='ssh_conn2', conn_type='SSH',
                    host='server_name', port='XXXX',login = 
                   'account_name',password = svcpassword))


        t2 = PythonOperator(
                task_id='Create_Connection',
                python_callable=create_new_connection,
                provide_context=True,
                dag=dag
                )


          t3 = SSHOperator(
                  task_id='RemoteCallTest',
                  command="R command",
                  ssh_hook = SSHHook().get_conn('ssh_conn2'),
                  do_xcom_push = False,
                  retries = 1,
                  dag=dag
             )



          t1 >> t2 >> t3

Answer 1

您需要利用会话包装器将更改持久保存到数据库

RouteBuilder

如何在代码中动态更新现有Airflow（1.9版）连接的参数？

1 个答案: