提取Azure Datafactory管道运行详细信息

Question

凭证= ServicePrincipalCredentials（client_id = client_id，secret = secret，tenant = tenant） adf_client = DataFactoryManagementClient（凭证，subscription_id） run_response = adf_client.pipelines.create_run（rg_name，df_name，pipeline_nm，{}） ＃监视管道运行 pipeline_run = adf_client.pipeline_runs.get（rg_name，df_name，run_response.run_id） 而（pipeline_run.status =='正在进行中'或pipeline_run.status =='已排队'）： #print（“ [INFO]：管道运行状态：{}”。format（pipeline_run.status）） time.sleep（statuschecktime） pipeline_run = adf_client.pipeline_runs.get（rg_name，df_name，run_response.run_id） print（“ [INFO]：管道运行状态：{}”。format（pipeline_run.status）） 打印（''）

        activity_runs_paged = list(adf_client.activity_runs.list_by_pipeline_run(rg_name, df_name, pipeline_run.run_id, datetime.now() - timedelta(1), datetime.now() + timedelta(1)))

Answer 1

活动运行与管道运行不同，如果要获取管道运行详细信息，请执行以下步骤。

1。Register an application with Azure AD and create a service principal

2。Get tenant and app ID values for signing in和Create a new application secret and save it

3。导航到门户网站中的数据工厂-> Access control (IAM)-> Add role assignment->将您的应用程序添加为角色，例如Contributor，详细信息紧随this。

4。安装软件包。

pip install azure-mgmt-resource
pip install azure-mgmt-datafactory

5。然后使用下面的代码根据输入的过滤条件查询工厂中的管道运行。

from azure.common.credentials import ServicePrincipalCredentials
from azure.mgmt.datafactory import DataFactoryManagementClient
from azure.mgmt.datafactory.models import *
from datetime import datetime, timedelta

subscription_id = "<subscription-id>"
rg_name = "<resource-group-name>"
df_name = "<datafactory-name>"

tenant_id = "<tenant-id>"
client_id = "<application-id (i.e client id)>"
client_secret = "<client-secret>"

credentials = ServicePrincipalCredentials(client_id=client_id, secret=client_secret, tenant=tenant_id)
adf_client = DataFactoryManagementClient(credentials, subscription_id)


filter_params = RunFilterParameters(last_updated_after=datetime.now() - timedelta(1), last_updated_before=datetime.now() + timedelta(1))
pipeline_runs = adf_client.pipeline_runs.query_by_factory(resource_group_name=rg_name, factory_name=df_name, filter_parameters = filter_params)

for pipeline_run in pipeline_runs.value:
    print(pipeline_run)

您还可以使用Run ID运行特定的管道。

specific_pipeline_run = adf_client.pipeline_runs.get(resource_group_name=rg_name,factory_name=df_name,run_id= "xxxxxxxx")
print(specific_pipeline_run)